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Preface 


The last couple of decades, society has undergone drastic changes, and digital tech- 
nology has brought about new ways of storing and producing information as well 
as new ways of communicating. These changes have also changed our views of 
learning, in ways that have resulted in new challenges for educational practices. 
Today, technology-enhanced learning is paired with distributed, collaborative, and 
self-guided learning processes. Examples are mobile learning, the flipped classroom, 
and web-based dialogues. Such phenomena presuppose so-called 21st Century Skills, 
skills that entail new demands and competencies for teachers as well as for school 
management. Therefore, educational research has developed in ways that deal with 
innovations as well as questions concerning how small-scale developmental projects 
can be scaled up (e.g., Halverson & Kelley 2017; Lim et al. 2019). 

In parallel to this development, texts have gained greater importance in society, 
and texts are crucial for learning in more or less all school subjects. At the same time, 
the meaning of “text” has changed. Today an extended text concept is used and we 
talk about texts as multimodal. Through this book, we want to highlight a new way 
of dealing with multimodal texts in parallel to presenting new ways of working with 
such texts in education. 

To be able to read and handle texts involves a lot more than the elementary 
reading competency to decode letters and words. Instead, text encounters involving 
new genres, new content areas, or new media imply new ways of approaching and 
reading texts. For students navigating between different text worlds, this has become 
more and more important. Also, for some time now, knowledge has been gained about 
the fact that different subjects’ areas use language in different ways (e.g., Halliday 
and Martin 1993; Schleppegrell 2004). However, not only verbal language (whether 
spoken or written) differs between genres and subjects, but other communicative 
resources too, such as images, diagrams, and graphs. Thus, a variety of resources 
can be subject-specific and result in potential challenges for the novice in the field. 

This book is intended for researchers, for teacher education, and for in-service 
teacher training. The motivation behind writing it has been to provide researchers 
and teachers with tools that can be utilized for texts in education. For this purpose 
we have developed a model for analyzing and working with multimodal texts. Also, 
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we want to highlight how a multimodal focus on texts can support students” making- 
meaning from texts, at the same time as such a focus can help students build up 
an awareness about the importance of text design, thus cultivating their multimodal 
literacy. 

Through this book, we want to shed light on the ways that texts can represent 
knowledge in different ways, and how the interaction between different text resources 
can be both a challenge and a support in meaning-making processes. This concerns 
wordings as well as other resources such as images and various graphic elements. 
Furthermore, through our model, we aim to demonstrate how teachers can work 
with texts in ways that can support students’ meaning-making through texts, and 
accordingly to deepen their knowledge in the content area in question. At the end of 
Part 1 of the book, we will discuss how our model can be used when working with 
students’ text production, and for assessing texts, paying attention to multimodal 
aspects of the text. 

Factual texts, whether paper-based or digital, whether shown on the computer 
screen or in different forms of applications for tablets or smartphones, are used in all 
subject areas. Therefore, this book deals with factual texts rather than literary texts, 
though multimodality can be relevant for such texts, too. 

A Swedish version of this book was published in 2014 (Danielsson and Selander 
2014) though this version of the book has been thoroughly revised, and in particular 
Part II, which consists of sample analyses of texts. Most texts in that part have been 
altered, to include more texts from other languages and cultures than the Swedish. 


Stockholm, Sweden Kristina Danielsson 
January 2021 Staffan Selander 
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Part I 
Multimodal Texts and Working 
with Multimodal Texts in Education 


Chapter 1 A) 
Introduction E 


Working with texts in deliberate ways is a foundation for supporting students’ subject 
learning, and their potential to demonstrate their knowledge in the subject. Therefore, 
it is central for all teachers to understand how texts work, and at the same time to be 
able to support their students’ competencies regarding how texts are designed and 
how to make meaning through texts. Thus, teachers in all school subjects have a 
responsibility in this regard, and teachers of the mother tongue cannot be expected 
to be the only ones responsible for developing students’ text competencies. Instead, 
teachers in all subjects need to deepen their own text competence. We hope that this 
book can be a support in doing so. 

Since text is the foundation of this book, and since this concept is used in both 
everyday language and academic language, this will be our point of departure in the 
following. 

An everyday understanding of text could perhaps be formulated as “written words 
on a paper or presented on a screen”. Academic definitions of text include such 
connotations, but research over the last decades has contributed to a more nuanced 
understanding of what a text is and how texts work. Therefore, the text concept has 
been extended in relation to “written words on paper”, now including other resources 
for meaning-making, such as image, speech, gesture, and so on. Each one of these 
resources can be seen as a text, provided that it corresponds to other aspects of text, 
such as consisting of a delimited message that can be understood in a specific context. 
Thus, in research and education today, we often deal with what is commonly termed 
an extended text concept. 

An extended text concept implies that texts can consist of a variety of commu- 
nicative resources that form a joint entity. Examples are written words! combined 
with various forms of illustrations, or a spoken text combined with gestures. Such 


lIn this book we use an extended text concept. Therefore, we try to be clear when referring to written 
or spoken words in texts—in opposition to e.g. tables, mathematical symbols, etc. Hence, in the 


following, we use concepts such as “written/spoken words”, “writing/speech”, or “spoken/written 
verbal text”. 
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texts are multimodal, and they are the focus of this book. We will discuss the term 
multimodal more thoroughly below. 

Since “text” today can involve so much more than just written words, it is important 
always to make clear how the text concept is used. Here we want to point out that 
there is no one correct definition of text. Instead, depending on the context or purpose 
for using the term “text” in a specific situation, the definition can vary. 

In this book, we restrict ourselves mainly to analyzing and discussing paper- 
based or digital visual texts, including all resources integrated in these (such as 
images, animations, or audio clips). Also, our focus is on pedagogic texts, that is, 
texts intended for more or less formal learning situations such as textbooks or Internet 
resources produced for educational purposes. Of course, texts that have not primarily 
been produced for educational purposes are also used in schools. We claim that our 
discussions of multimodal texts, as well as our model for working with such texts, 
function well for any kind of text. However, there are good reasons to base an expo- 
sition like this on texts that are typically used in education. Teachers’ and students’ 
oral texts will only be commented on briefly, not because we are not interested in 
them, or because we find them unimportant, but for delimitation reasons. We will 
also look more closely at a couple of texts made by students, in order to discuss 
multimodal aspects of students’ text production. 

The aim of this book is to provide the reader with an analytical framework for 
multimodal texts, and to suggest ways of working with multimodal texts in the class- 
room practice. In Sect. 4.5, Model for working with multimodal texts in education, we 
present a model for working “hands-on” in classroom practices. Based on a number 
of examples from real texts, we guide the reader through the model, suggesting ways 
of approaching texts to be better able to note how the texts are constructed, and what 
might need to be specifically focused on in the classroom. In the second part of the 
book, the model is implemented on a number of texts from different subject areas 
and intended age groups as well as from different countries. In relation to these text 
analyses we also discuss classroom implications. 

The selection of texts in the second part of the book has been made with the intent 
to use the model for a variety of school subjects, and at the same time the texts have 
been taken from different countries. Apart from Sweden, which is where we live 
and work, text examples have been taken from other countries, such as Portugal, 
Singapore, and Chile. The text examples are taken from different subject areas and 
intended age groups, even though most of the texts are produced for students in 
upper elementary (grade 4—6) and lower secondary school. Factual texts geared to 
younger students are usually quite similar to those produced for grade 4—6 students, 
even though the amount of written verbal text is smaller, at the same time as the 
illustrations can be less complicated. Therefore, the examples chosen can also serve 
well for teachers working with students in lower grades. Correspondingly, texts for 
secondary school students can be seen as parallel to texts used in upper secondary 
school, even though texts for the later grades will be more demanding. 

The majority of the texts are printed textbooks that are relatively widely used in 
the respective countries. For example, the Singaporean textbooks are obligatory in 
Singapore, and the Swedish textbooks are books that are relatively widely spread 
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in Sweden, where each school decides what textbooks to use. The choice to focus 
on printed media is connected to the fact that printed books are still important in 
many schools. Also, many of the digitally produced textbooks today are still mainly 
based on printed versions of the texts, though in the near future we will probably see 
changes in this regard. 

It is important to stress that the kind of close reading we perform in relation to 
our analyses is not done to draw attention to limitations or negative sides of the 
texts. Instead, our intention is to point out challenges that can arise in relation to 
any multimodal text, something which also justifies conscious ways of working with 
texts in all school subjects. 


1.1 Multimodal Texts 


As mentioned, an extended text concept is widely used today, and more and more 
researchers and teachers are embracing such a text concept, understanding that all 
texts are multimodal, that is, they are composed of a variety of sign systems, or 
semiotic resources, such as words, diagrams, graphs, photographs, or various kinds 
of symbols (e.g. Kress and van Leeuwen 2006, also see Berge 2001, for a discussion 
on the text concept). It is also common to use various kinds of graphically marked 
text objects, such as headings or ingresses. Web pages, magazines and the daily press 
are filled with such resources. Particularly in educational texts, there is an abundance 
of graphically marked text objects for different purposes, such as text boxes and 
bullet lists. 

Thus, a number of features make texts multimodal, not only the inclusion of 
images. The layout and the ways in which different parts of the text carry meaning, 
is also important to understand. Potentially, any part of a text can carry meaning in 
specific ways. Sometimes the information across different text elements is contra- 
dictory, sometimes the elements complement each other. At times verbal text (i.e. 
the written words) is the most important semiotic mode, sometimes the image. Each 
page thus offers different challenges for text interpretation, not least for students who 
are expected to make meaning of complex content from the text. 

It is quite common that students in lower grades are encouraged to express them- 
selves in both images and words. A number of studies, however, have revealed that 
it is unusual for teachers to comment on what is expressed in these images. This is 
true of narrative texts as well as more descriptive, factual texts. Usually the teachers 
regard them as some kind of ornament, focusing on aesthetic aspects of the image at 
a quite basic level, such as commenting on the image as, for instance, “nice”, even 
when the images indeed carry specific and important meanings (e.g. Peled-Elhanan 
2015). The use of such general comments might be explained by the fact that many 
teachers experience a lack of tools for giving constructive feedback in regard to other 
aspects of the images, and for multimodal aspects of texts. We hope that this book 
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will provide such tools. Since our discussions and analyses are based on the rela- 
tions between text design (e.g. Kress 2010) and content, aesthetic aspects will be 
disregarded in the following. 


1.2 A Historic Perspective on Multimodality 


During different historical eras, people have used various resources in their commu- 
nication. In ancient Greece and Rome, speech was the predominant resource for 
communication. Rhetoric, which was the analytical and educational instrument 
of that time, highlighted that the speaker needed to make the right choices when 
addressing the listener, and to shape their speeches to suit the specific purpose. Also, 
according to rhetoric, the speaker could utilize a number of other resources than 
speech, such as more or less formalized gestures to underline certain aspects of the 
speech, variations in tempo and intonation, or placement in the room. 

When people started to use written language in different contexts, the function 
was to keep account of money, to make new laws, to describe historical events, or 
to celebrate a certain person, for example a Roman emperor who had won a battle, 
or a Nordic viking who had traveled far away and gained a fortune. During the 
Middle Ages, a system for large-scale copying was developed to copy handwritten 
biblical texts. Here, verbal text was combined with different forms of illustrations. 
In addition, the interior of the medieval church is an example of a multimodal text, 
where biblical stories were illustrated on the ceiling, on the pulpit, or as carved 
images on the baptismal font. 

The development of the letter press led to the possibility of mass distribution of 
texts. When a market for books and papers arose, information could be stored and 
spread to a great many more people than had previously been possible. This situation 
also challenged different kinds of power relations in society. The printed word also 
repositioned the collective memory, and when public schools were introduced, the 
printed word had a special status concerning learning. To be knowledgeable was 
more or less equivalent to being able to memorize printed words. 

Well into the twentieth century, school textbooks in Western society were based on 
writing, and they were quite often expressed in a narrative style. These written texts 
were sometimes illustrated with fact-based images or reproductions of paintings. 
From the middle of the last century, however, we have seen a change in the relation 
between writing and image, and image has become more salient and important in 
pedagogic texts (e.g. Bezemer and Kress 2008). 

Thanks to the breakthrough of digital media at the start of the new millennium, 
various kinds of educational games and applications for digital learning have come 
into play in education, at the same time as mobile phones and tablets can be used 
in more advanced ways (Arnseth et al. 2018; Devlin 2011; Egenfeldt-Nielsen et al. 
2011; Lindstrand et al. 2016; Selander 2008; Squire 2011; Steinkuehler et al. 2012). 

To sum up, writing and image have been used in different ways through history, 
and different communicative resources have played different roles in different kinds 
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of texts. Today, a plethora of media are used in text production, and texts generally 
contain a variety of semiotic resources. Hence, it is important to pay attention to 
what kind of information is given through speech, writing, or image—or to put 
in another way: in what medium and through what semiotic resources is content 
expressed? Not least from an educational perspective, it is crucial to be able to discern 
how information is expressed and to realize how students are given opportunities 
to use different media and semiotic resources, in order to learn and to show their 
knowledge (Insulander et al. 2017). As a consequence, the forms for assessing and 
testing knowledge in educational contexts need to be developed in ways that allow 
for more resources than the written or spoken word. 
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Chapter 2 A) 
Multimodal Texts and Literacy get 
in a Digitized World 


When working with multimodal texts in education the starting point must be the ways 
in which contemporary texts are constructed: they are multimodal and composed 
of various textual elements, something which can be challenging for the learner 
who is not yet familiar with a specific content area or text type. If teachers guide 
their students into an understanding of the ways in which texts are designed and 
how the information structure can be understood, they can harness their students’ 
content learning. Other central aspects for teachers working with texts in education 
include developing a greater understanding of student-created multimodal texts, and 
developing assessment criteria that do justice to such texts. 

The digitized media landscape affects society in several ways. In school contexts, 
there are numerous examples of how contemporary text and media worlds affect the 
ways in which educational settings are designed. One such example is the flipped 
classrooms where teachers provide their students with video-recorded outlines or 
lectures, or provide other learning resources digitally for the students to prepare 
before actually meeting in the classroom. During lessons in the physical classroom, 
time and effort can then be devoted to discussions, experiments or other types of 
activities. The access to digital resources affects the ways in which both teachers 
and students use their time, individually or in groups. However, regardless of the use 
of paper-based (textbooks and other types of texts) or digital media, students might 
need guidance in how to approach the texts. 


2.1 Texts in and Outside of Schools 


In contrast to the situation that prevailed when the authors of this book went to school, 
when there were still school subjects in which texts were not used at all, or to a very 
little extent, or where the subject content was mainly based on practical training, or 
trying out different techniques (such as physical education or art), theoretical discus- 
sions have become an integrated aspect of most subjects (to our knowledge, this is the 
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situation not only in the Scandinavian countries, where we have our main experience). 
However, “theory” does not necessarily have to be equivalent to “written texts”, but 
even so, written texts have become increasingly important in many different subjects, 
as is evident in the curricula in many countries. This change is also evident outside 
of school contexts; in increasingly more workplaces employees are expected to use 
writing in quite complex ways, even in contexts that we might perceive as mainly 
manual (e.g. Karlsson 2009; Nikolaidou 2014). Our spare time, too, is filled with 
various text activities, from shopping lists and the planning of leisure activities to 
text messaging, Facebooking or Twittering (e.g. Barton and Hamilton 1998; Davies 
2012). Thus, we live today in a text-based society with partly new types of text 
cultures. 

To encounter a new disciplinary content in school always involves new ways of 
communicating. A lot has been written about the ways in which texts are structured 
linguistically in different content areas (e.g. Schleppegrell 2004; Martin and Rose 
2008). Alongside this growing awareness, researchers have stressed the importance 
of working with language in conjunction with content in all school subjects. Such a 
way of dealing with disciplinary language (see Shanahan and Shanahan 2012, for a 
discussion on “subject literacy” vs. “disciplinary literacy”) has been a central aspect 
in genre-based pedagogies, where one concern has been to find ways to support 
students in ways that promote their possibilities to become part of the discourse of 
different subject areas and the ways in which ideas are communicated within them 
(Christie and Derewianka 2008; Fang and Schleppegrell 2004). Regarding the use of 
other meaning-making resources than verbal language (written or spoken), however, 
there are obvious differences between disciplines too. Not only terminology or the 
ways in which the verbal text is structured differ between, for instance, a book 
spread in history and one in chemistry. Visual resources such as images, diagrams, 
and tables are just as important concerning textual choices. Therefore, learning a 
new subject and becoming part of the discourse in that subject involves dealing 
with many different meaning-making resources. Furthermore, it is essential to be 
conscious about when and how one meaning-making resource is the most efficient 
and appropriate to use (cf. affordance, below). 


2.2 Text and Literacy—Changing Concepts 


Parallel to the extension of the text concept, literacy has an extended definition 
today. Within the tradition of New Literacy Studies (NLS) (Barton 2007, Barton and 
Hamilton 1998) it has long been claimed that reading and writing are not delim- 
ited competencies for an individual to develop once and for all. Instead, from the 
perspective of NLS, encounters of new discursive areas entail new literacy practices. 
Therefore, during the course of our lives, we will be presented to new literacy chal- 
lenges again and again. So far, New Literacy Studies have mainly focused on literacy 
practices in people’s everyday lives (e.g. Barton and Hamilton 1998) or at workplaces 
(e.g. Karlsson 2009; Nikolaidou 2014). However, the question about literacy practices 
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in different contexts is as relevant for the school context, and not least in Scandina- 
vian contexts, researchers have been inspired by this way of approaching texts and 
text use. The question of literacy practices within different fields of knowledge is 
also relevant from the multimodal perspective adopted in this book. 


2.3 Towards a Digital Text World 


The digitized world of today has brought new possibilities regarding text design and 
text production (e.g. Selander 2015; Sheridan and Rowsell 2010). To put it simply, 
anyone who uses digital media in their text production also becomes a text designer. 
A slightly more complex perspective on digitization is to view digitized texts as texts 
that involve new possibilities, such as integrating links to other texts or web pages, 
to use color for various purposes, to make animations, and to integrate sound or film 
clips in the text, and so on. 

Digitization has also enabled new forms of mobile learning. One example is 
the use of mobile phones to perform mathematical calculations or to analyze water 
quality during excursions in nature, while simultaneously being able to communicate 
with students in other countries working with similar themes (Eliasson 2013; Nouri 
2011). Thus, by using digital techniques, online communication is enabled, where 
other students can be invited to comment on the types of texts in use. Therefore, both 
texts and the context of creating texts are extended in quite radical ways. 

Traditional paper-based texts—in which sentences are connected to paragraphs 
and paragraphs are connected to a whole—can be seen as more or less linear and 
with clear boundaries: we know where the text begins and where it ends. A digi- 
tally produced text can be linear in the same fashion, but the medium enables other 
possibilities that open up for a more flexible reading order. For the reader, therefore, 
the reading process differs from traditional texts. In a digital text, the question of 
beginning or ending is not self-evident. For example, should linked texts—possibly 
involving yet other links—be considered part of the text or not? A Facebook page 
is constantly under revision (at least with an active user), and the owner as well as 
other participants can add features such as text, images, or links. 

Not the least among young people, applications such as fan fiction are popular, 
that is, websites where fans can write stories based on existing fictional worlds, 
such as popular novels, computer games, or television series, and where participants 
encourage each other and comment on each other’s texts. In such communities, 
anyone who is able to create a text with a style similar to the original author can 
be expected to be particularly successful. In Chap. 5, “Cultivating Students’ Text 
Creation”, we will discuss digitally created texts in relation to text compilation and 
the use of references to sources (that is, “who owns the text?’’). In school contexts, 
compilation texts are often regarded as problematic. In relation to, say, fan fiction, 
however, the question of ownership of texts is a difficult one, and maybe it is actually 
an irrelevant question altogether? This is how WideEyedDreamer01 at fanfiction.net 
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comments on his/her Harry Potter text: “Disclaimer: I don’t own the Harry Potter 
series, it all belongs to JK Rowling. Enjoy and Review!” (fanfiction.net 2013). 

When students work with digital media, one might expect a spontaneous use of 
multimodal possibilities, and the integration of various types of visual resources. 
Yet this is not always the case. A study from chemistry classrooms revealed that the 
digital possibilities were mainly used to create artistic headings (Danielsson 2011). 
In another project, students in a physics classroom complained when they were 
asked to use their tablets when drawing, commenting that it is easier to use pen and 
paper (Danielsson manuscript). In the chemistry classroom, students were relatively 
unaccustomed to using digital resources for laboratory reports. Furthermore, in many 
ways it is still easier to make drawings by hand. 

Digital versions of textbooks are often examples of texts where the digital version 
is close to the printed one and where the dynamic possibilities are utilized only to a 
little extent. In previous textbook analyses, we have noted that textbook publishers 
quite frequently launch a digital version in parallel to a paper-based version of the 
same book (Danielsson and Selander 2014). Sometimes, but not always, the digital 
version includes the possibility for users to listen to the text by clicking on sections. 
Another option is to provide the user with the possibility to navigate between sections 
through links in indexes, or to include search functions, or the possibility to make 
digital notes in the text. In many of the texts we have analyzed, illustrations on 
the book spreads were identical between the paper-based and the digital version. 
Perhaps one might have expected greater differences between the versions given the 
potentials for digital media. However, it is likely that in the near future we will see 
rapid development in this area. 

Today it is common for students to use the Internet to search for information, at 
least in schools where students are given access to computers or tablets. Of course, 
there are great differences as to the economic and practical conditions both between 
and within countries. For younger kids, there is a plethora of educational learning 
sites with links to other digital resources. In Part II of the book, we comment on one 
such site, produced by National Geographic. 

An interesting Swedish study was performed by Ulrika Nemeth (2011), who 
investigated the use of web pages among upper secondary students during school 
lessons. In the classroom there were discussions about the reliability of sources, for 
instance in relation to the producer of the page. What was particularly interesting 
from a multimodal perspective, however, was that when students were searching the 
Internet for information related to assessments, the students themselves discussed 
reliability from other perspectives, such as layout. From these discussions it was 
obvious that the students considered pages with an inconspicuous design to be more 
reliable than colorful pages, or pages with many animations. 

The use of film clips from YouTube, produced for formal learning, is another 
example of digital resources utilized for pedagogical reasons. What is striking in 
this regard is the variety of presentational styles in such clips. There are quite a few 
examples where, for instance, PowerPoint notes, or video-recorded blackboard notes, 
are commented on through a relatively traditional spoken lecture. These types of 
recorded presentations can be seen as parallel to digital versions of printed textbooks, 
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where it is mainly the media of presentation that has been altered, but not really the 
form of presentation. In, for example, the area of natural sciences, on the other 
hand, the technical possibilities of digital media have been used to produce relatively 
advanced animations (for instance regarding the relation between the sun and the 
earth), video-recorded experiments, or video clips like “Chemical Dance Party”, 
with students acting as different chemical substances to illustrate the substances” 
tendencies to react with other substances.! 

The changes of the conditions for texts and text-work in educational settings also 
include the use of games and simulations (Arnseth et al. 2019; Brooks et al. 2021; 
Gallagher 2015; Mørch and Thomassen 2016). One example is the Swedish game 
Minecraft, which we comment on in more detail in Chap. 13, “Educational Games 
and On-line Resources”. 

When using games for learning purposes, it is important to make the learning 
goals and learning objects explicit to the students. Also, the teacher can combine the 
use of games with other tasks, “outside” of the actual game to benefit from the joint 
engagement in the classroom. 

In an international developmental project, researchers and teachers from Australia, 
Ireland, and Sweden collaborated in developing a game for collaborative problem 
solving and on the assessment and teaching of “21st Century Skills” (Nouri et al. 
2017). In this game, students worked in pairs to solve a joint problem. The idea was 
that the two students had access to different information about the same problem 
and they needed to collaborate to get access to all information. Also, they needed to 
collaborate to find out what the problem actually was about and what information 
was relevant for solving the problem. To adapt the games to different educational 
contexts, researchers and teachers in the different countries adapted the games to be 
in line with the curricula and the specific learning goals in each country. 

The teachers noticed some specific problems related to working with the cases 
and the assessment of the cases: one was that information about the group work 
had to be very precise in order for the students to be able to handle the cases; the 
other problem was that it was not evident which school subject “owned” a problem 
like “the pollution of the Baltic Sea”, and was thus responsible for the assessment: 
was it Social Sciences, Chemistry, Physics, or History, or a combination of these 
subjects? In sum, the development of these games point towards interesting new 
possibilities: (i) to develop small-scale and locally adapted educational games, (ii) to 
include game-oriented elements in the learning activities, (iii) to use the opportunity 
to work collaboratively between subjects. 

Hence, just as there is no typical paper-based text, there is no typical digital 
text. In educational contexts, the same challenges are connected to digital texts as 
for traditional, paper-based texts. Therefore, students might need different kinds of 
support when choosing and making meaning from digital texts. Thus, it is important 
for teachers always to consider the function of the various ways of (re-)presenting 
content, and how different choices, such as animations of ionic bond dancing at a 
“chemical party”, might be challenging in students’ meaning-making process. 


LA YouTube search for “Chemical Dance Party” results in a number of such films. 
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As mentioned, in educational contexts, the flipped classroom, or flip teaching, is 
growing. Using such ways of designing teaching is one way of employing digital 
media to be able to use time in the classroom in partly new ways. Often what students 
have traditionally been provided with in the classroom—such as a theoretical back- 
ground to a content area—is now given through digital resources outside the class- 
room, and the students get access to these teaching resources through, for example, 
a blog. An example of flip teaching is when students are asked to view recorded 
lectures or PowerPoint presentations produced by the teacher, or freely accessible 
on the Internet. In that case, students can use the teaching resource in ways that 
fit their respective interests or learning conditions, and the classroom time can be 
used for activities like problem solving, group discussions, or hands-on activities, 
such as experimental work. When organizing learning activities in such ways, the 
time in the classroom can be used to focus on implementing the content that has been 
provided through digital resources, and for activities where the teacher interacts with 
the students in various ways. 

The underlying idea behind the flipped classroom is for teachers and students to 
use classroom time for communicative activities or investigations of different types, 
and for the teacher to support the students in various ways. One benefit—apart from 
being able to utilize joint classroom time for communicative activities—is said to 
be that students often find classroom activities more meaningful. However, such a 
setup requires that students both have access to and are able to handle digital media, 
and also that they are able to take responsibility for their own learning outside the 
classroom. 


2.4 Summary 


In this section we have mentioned a number of changes regarding text practices in and 
outside of school. All of them affect curriculum documents and ways of formulating 
requirements regarding what counts as knowledge and how to assess it. 

The digitized society has led to new possibilities and challenges regarding text 
design and text production. Everyone who uses digital media has also become a text 
designer. Furthermore, digital texts provide us with new affordances, such as the 
possibility to incorporate links and to add audio- and film-clips in texts. The Internet 
has also led to a new culture of sharing. One effect of such a sharing culture is that 
the limits of a certain text are no longer evident, and it is easier to integrate other 
texts in new texts, something which has consequences for education (Selander 2017). 
The digital development has also created new forms of teaching, such as the flipped 
classroom. 
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Chapter 3 R) 
Semiotic Modes and Representations get 
of Knowledge 


When organizing our understanding of the world around us, we use semiotic resources 
(e.g. Kress 2010). Semiotic resources are resources that we use to organize our 
understanding of the world and to make meaning in communication with others, or to 
make meaning for ourselves.! When these semiotic resources are used in systematic 
ways they form semiotic modes. Examples of semiotic modes are color (when color 
is used in more or less conventionalized ways), spoken or written verbal language, 
images, or gestures. A semiotic mode in itself can actually be viewed as multimodal, 
for example an image where color is used as one meaning-making resource. Semiotic 
modes can also be combined differently in different media—such as paper-based or 
digital media. In all social communication we combine semiotic modes. Thus, social 
communication is always multimodal (e.g. Jewitt et al. 2016). 


3.1 Knowledge Representations 


One might assume that we can choose any available semiotic mode for making 
meaning, and that it is possible to express “the same” content through any semiotic 
mode. From such a perspective, the choice of semiotic mode would rather be based 
on aesthetical or practical choice. A more philosophical question is whether it is 
actually possible to separate content from form. If we imply that all knowledge must 
formulated somehow, and that all knowledge is represented through some kind of 
system for meaning-making, form is crucial for deciding what aspects of the content 
are given precedence. For an expert of a field, the ways in which content is presented 
are of minor importance. For the novice entering new knowledge areas, however, the 


'The difference between semiotics and social semiotics is that semiotics emphasizes the relation 
between sign and meaning, while in social semiotics social conventions and situational factors are 
given greater prominence (Kress 2010). 
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ways in which content is presented can be crucial for the learning process (also see 
Jewitt 2008; Selander and Kress 2010; Danielsson 2016). 


3.2 What Semiotic Modes “Count” in Schools? 


The fact that we can use many ways of communicating content, and that all written 
texts to some extent are multimodal, can be contrasted to the text traditions in schools, 
where the written word is given most prominence, as the resource that is still valued 
and “counts”. 

By focusing only on the words in student texts, there is a risk of missing important 
aspects of what is actually said. In a project on the text worlds of pupils in primary 
school, Charlotte Engblom (2010) made an interesting point regarding what counts 
for teachers and pupils, and that their interests can sometimes be contradictory. In 
one classroom activity, when students were asked to produce a text on the computer, 
one pupil made a conscious spelling “mistake” on the word mum, to allow the words 
in the sentence to be placed in a way that made the image appear at the right place 
on the page. In this case, the visual appearance of the different text resources (image 
and words) was considered more important than correct spelling. The problem for 
this student was not spelling, but instead how to handle the digital tool, in this case 
to make the words appear at the right place without manipulating the word length. 

With regard to different views on texts and what counts, traditions vary in different 
school subjects. In more practically oriented areas, such as crafts or arts subjects, the 
tradition of viewing the written word as the most important resource is less prominent. 
At the same time, in such areas there is a risk that the plethora of multimodal texts 
that are actually utilized might be overlooked, for instance sewing instructions or 
drawn plans. However, even subjects that we regard as highly theoretical, such as 
the natural sciences, have long traditions of using an abundance of visual resources 
and representations, such as images, diagrams, or three-dimensional models. 


3.3 Semiotic Mode and Meaning Potential 


One aspect related to semiotic mode is their aptness in relation to different types of 
content. Spoken or written words are specifically suitable for expressing temporal 
aspects (first this happened, and then this) or issues of cause and effect (x happened 
because of y), while images are better apt for spatial relations (e.g. Kress 2003). This 
does not mean that we cannot use images to express temporal relations; if we want 
to do that we can use more or less culturally established resources such as arrows, 
reading order between a series of images, etc. 

The choice of resources for meaning-making is linked to the notion of modal 
affordance (e.g. Gibson 1977; Kress 2003), in other words the “meaning-making 
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potential” or the “potentials and limitations” of certain semiotic resources or modes 
in a specific situation. 

The term affordance can be illustrated by the variety of semiotic resources that 
students come across when a natural phenomenon like the atom is explained in 
chemistry classrooms (Danielsson 2016). For example, in textbooks, combinations 
of writing, including chemical or physical symbols, and images are used, and in 
their expositions, teachers use words (spoken or written), written symbols, images 
(drawn on the board or projected on a screen), and gestures. In words, metaphors 
like “electronic cloud” or “electron shells” are used, or the atom is described through 
analogies, for instance by comparing electronic orbits with the planets’ (electrons) 
movements around the sun (atomic nucleus). 

The choice of semiotic resource for meaning-making is never made at random. 
Instead, more or less conscious choices are made, based on available modes or specific 
resources in the meaning-making situation. The choice is also dependent on what 
we want to say, and to whom, depending on the affordance of different resources. 
Also, the choice is reflected by what aspects of a phenomenon a certain representa- 
tion can be used for (e.g. Danielsson 2016, Tytler et al. 2013). For the novice, the 
intimate connection between content and resources chosen for representation has 
consequences for learning: to learn what signifies an atom cannot be separated from 
learning how the atom can be represented. This connection is commented on by 
Diana Laurillard thus: “Knowledge technologies shape what is learned by changing 
how itis learned” (Laurillard 2012, p. 3). She illustrates this statement by an example 
from a course in Business Studies: when the students are expected to read and write, 
their analytical competencies are supported, when role play is used, their experi- 
ences and their understanding concerning interpersonal relations, and when they are 
using Excel sheets, experiments and calculations of different economic parameters 
are supported. The given—and conventional—representations have consequences 
for what is recognized and assessed as relevant knowledge in the subject.” 

Thus, different resources have different potentials for meaning-making. The 
different particles of an atom can be illustrated through a drawn image, and at the 
same time you get an approximate idea of how the different particles are related: 
that protons and neutrons form the nucleus, and that electrons revolve around the 
nucleus. Yet, when we draw an image, we have to decide where to place the different 
parts in relation to each other, and we have to decide their relative sizes. 

A drawing of the atom will always be a simplification, and in fact a falsification, 
for example since the electrons are so extremely small in relation to the protons and 
neutrons in the nucleus, something that cannot be illustrated on a limited surface such 
as a textbook page or a board in the classroom. Furthermore, the electrons have to 
be placed somewhere. They are often drawn in distinctly delimited orbits, although 
there are in fact no such distinct orbits. Instead, the electronic shells one talks about 
in science are more of an illustration used to indicate the plausibility of the position 
of an electron at a given moment. 


?Kress and Selander (2012) discuss this in terms of what is recognized as “signs of learning” within 
“cultures of recognition and assessment”. 
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Fig. 3.1 Teacher's illustration of the atomic model in a secondary school chemistry classroom 
(from Danielsson 2016, with permission from the author) 


You need only a quick look at the illustration in Fig. 3.1 to get a general image of 
the different particles of the atom, and their approximate relations to each other. On 
the other hand, through words, it is possible to clarify that the electrons are not placed 
in clearly defined orbits, and that the exact placement is always arbitrary. Through 
words and mathematical symbols you could also describe, for example, the relative 
sizes of protons and electrons. With multimodal ensembles, such as combinations of 
words, images, and other resources, a relevant (though still simplified) image of the 
atom as a natural phenomenon can be given. 

Already when students start school, they have some level of knowledge—most 
probably implicit—of what is best said through what semiotic mode. Figure 3.2 is 
an example of how a Norwegian pupil in first grade uses a combination of writing 
and image when creating a text after an excursion to the sea (Sjøhelle 2013). In 
that text, the student has drawn a crab that he found during the excursion. For this 
student it is obviously an important piece of information that crabs move sideways. 
However, to depict such a movement in an image is difficult, and he comments on 
that in writing: “crabs move sideways” (No. kraber gar sidlengs). School can benefit 
from the fact that even young students use different modes in accordance with their 
affordance when creating their own texts and when working with pedagogic texts in 
the classroom teachers can connect to this implicit knowledge. 

Resources used in meaning-making have a culturally established meaning poten- 
tial which embraces both the offered meaning (for example the intention of the 
illustrator or the author) and the perceived meaning (what the reader of the text pays 
attention to). Not the least from an educational perspective, it is important to note 
that in the actual meaning-making situation it is not always the case that the intended 
meaning coincides with the perceived meaning. Instead, the reader’s interest, as well 
as the reader’s previous knowledge in the area, can direct what meaning is actually 
made from the text. 
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Fig. 3.2 Student text (year 
1) after an excursion to the 
sea (Sjøhelle 2013, p. 114, 

re-printed with permission 

from the author) 


3.4 Implications for Education 


From an educational perspective, it is a great asset to have an abundance of semiotic 
resources to choose from when engaging in various aspects of content. Yet we cannot 
take it for granted that the students will be able to handle the different resources 
without guidance, or to be able to see their possibilities and limitations. If we stick 
to the example of the atomic model, we can imagine that for anyone who already 
knows the structure of the atom, and the characteristics of this natural phenomenon, 
it can be functional to compare the atom with an apple (an analogy sometimes used in 
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Swedish speaking classrooms?) without pondering about what the fruit pulp could be 
analogous to. Furthermore, the expert knows that there are no prefabricated “shells” 
for electrons to place themselves upon. However, such analogies can give false ideas 
to anyone who is not familiar with the content, such as a student who in an interview 
expressed the idea that “the shell becomes empty” when ions are formed, with valence 
electrons “leaving” an outer electron shell. 


3.5 Summary 


In this chapter, we have discussed semiotic resources in terms of the resources we 
use to communicate and to organize our understanding of the world around us. When 
semiotic resources are used in systematic ways, these can be seen as semiotic modes. 
Examples of semiotic modes are verbal language (written or spoken), gestures, and 
color (when the color in itself carries meaning, such as red for heat or to depict 
“stop!’”). 

What semiotic resource to use—and in which semiotic mode—is connected to 
the content that is communicated: you cannot say exactly the same thing through any 
semiotic resource. Semiotic resources or modes are said to have different affordances, 
or potentials for meaning-making. 

The ways in which a text is designed through different semiotic resources are 
particularly important for the “novice” who is about to enter a new domain of 
knowledge. From an educational perspective, we cannot take it for granted that our 
students are able to handle the different multimodal resources without guidance, or 
to fully understand their meaning-making potential (or affordances and limitations). 
By working actively with the students and making visible to them how texts are 
structured, we can support them in reading the texts as genres of texts in specific 
content areas, and consequently make it possible for them to deepen their knowledge 
in the particular subject content. 

The extended text concept as well as the understanding of multimodal texts chal- 
lenge the dominant school tradition, where writing has been seen as the main resource 
and the resource that is valued and “counts” as representation of knowledge. It also 
challenges our more traditional ways of assessing knowledge primarily by the use 
of “written, verbal language”. 


3In Swedish, the word for nucleus (kärna) is the same as for seed (kärna), and shell for atomic shell 
is the same word as peel (skal). Therefore, this analogy is possible in a Swedish-speaking context 
(see Danielsson et al. 2018). 
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Chapter 4 ®) 
Working with Multimodal Texts creek 
in Education 


4.1 Who Can Read and Write? 


The insight that different texts and genres make different demands in the meaning- 
making situation must lead to an extended view of reading and writing as meaning- 
making activities, and of literacy development for the individual. From that perspec- 
tive, the development of literacy does not consist of “learning to read and write”. 
Instead, an important aspect is to develop textual repertoires to be able to make 
meaning through different texts, including repertoires concerning writing as well as 
other semiotic modes—in different genres and in different subject-specific contexts. 
Furthermore, the development of textual repertoires concerns a variety of texts, 
from writing in paper-based media to highly multimodal, hyperlinked texts in digital 
media. 

Today, all teachers must handle multimodal texts, and they need to have an aware- 
ness about the ways in which different resources for meaning-making work. However, 
they also need to develop teaching practices that involve strategies that enable ways 
of handling texts that are perceived as challenging. From this point of view, meaning- 
making through texts must develop continuously. There is no exact moment when we 
can say that a certain student “has learned to read” in an absolute sense, something 
which makes partly new demands of teachers in all disciplines. 


4.2 The Expert and the Novice 


As mentioned, each semiotic mode has a more or less culturally established potential 
for meaning-making. However, readers note and interpret different aspects of a text 
depending on their previous knowledge, specific interests, understandings of the 
situation, genre knowledge, etc. This situation applies both to anyone knowledgeable 
in the field and to those for whom the field is new. It is especially important here to be 
aware of the fact that someone with a great knowledge of the field might not realize 
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the potential challenges in texts or other learning resources. The expert in the field 
will, without being conscious about it, fill in potential gaps, and weed out extraneous 
information. Moreover, it is doubtful (or rather, it is implausible) that the offered 
meaning from the perspective of the expert will be the same thing as the perceived 
meaning from the perspective of the novice. 


4.3 Curriculum Documents 


Every country compiles a curriculum in line with its own traditions, but also inspired 
by the development in other countries. Thus, we can find examples of curricula with a 
clear emphasis on multimodal texts, even though most countries do not yet explicitly 
require multimodal competence as a specific competence in their curricula. 

At the same time global research, as well as new educational practices concerning 
how to represent knowledge and work with learning in schools, obviously change 
the learning landscape. Therefore, a multimodal (and multimedial) understanding 
seems to be of increasing importance. 


4.4 Text Talk in Education 


It is common for teachers to work with students” prior knowledge of the subject 
matter when introducing new content in class. Working with prior knowledge can 
involve giving an overall introduction to the content area and working with central 
concepts, or connecting to the students” previous knowledge of the area in various 
ways. To work with students” previous understanding in such ways can function as 
a support for many students, not least to facilitate their understanding of a complex 
content. Yet, if the teacher does not also support the students in how to grapple with 
the actual texts that they are supposed to make meaning from, they might never need 
to get to grips with texts. This in turn can have negative consequences for the students 
as they might lack essential tools later on, when they will have to deal with different 
kinds of texts independently. 

Numerous classroom studies over the years, from different subject areas and 
school years have revealed that teachers seldom engage students in discussions 
focusing on how texts within the area are structured and why (Danielsson 2010; 
Digisi and Willett 1995; Edling 2006; Wellington and Osborne 2001). It appears 
as if it is often taken for granted that anyone who has broken the written code and 
reached some level of reading fluency should not need any guidance as to how to make 
meaning from or produce different kinds of texts. An exception is of course writing 
education within the mother tongue subject. Also, there are examples of schools 
focusing on language and content, where teachers in the mother tongue collaborate 
closely with teachers in other subjects, for example in schools working with the genre 
pedagogy. More often, however, if any collaboration takes place between subjects, it 
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often concerns a general level, with no focus on the chracteristics of texts in different 
subjects. 

Furthermore, research has shown that even within subjects where an abundance 
of resources are central for meaning-making, such as images, abstract models, math- 
ematical symbols, etc., teachers tend not to highlight the specific demands inherent 
in such texts (Danielsson 2010; Lgvland 2010; Öman and Sofkova Hashemi 2015). 
Both teachers and students seem to assume that images always support the content, 
and that images in themselves cannot be a challenge in the meaning-making process. 
Furthermore, teachers appear not to draw any specific attention to the multimodal 
aspects of student texts, for example when students create PowerPoint presentations. 

There may be various ways of explaining why so many studies reveal the same 
pattern. It could be that we tend to take certain things for granted, as we commented 
on in relation to the semiotic modes that are valued and “count” in school. It could 
also be the case that teachers lack tools for talking about and assessing multimodal 
aspects of texts (Bearne 2009). Teachers might want to highlight multimodal aspects 
of texts, but they do not really know how to do it. We hope that our discussions 
about multimodality and the model for working with multimodal texts in education 
can function as a support for teachers wanting to develop their teaching in such a 
direction. 

Before presenting our model for working with multimodal texts, we will comment 
briefly on some other models that have been developed for working with texts in 
education, although primarily with writing (i.e. written words). 

A number of these models are based on methods for working with reading compre- 
hension, such as Questioning the Author, which is a scheme developed by Isabel Beck 
and Margaret McKeown for the reading of literature (e.g. Beck and McKeown 2006), 
which has also been adopted for working with factual texts. 

The basis of genre pedagogies (e.g. Christie and Derewianka 2008) is that an 
explicit focus on text structure in class will harness students in their text encounters. 
Within genre pedagogy, both the form and the function of texts in different subject 
areas are emphasized. Thus, the focus is both on text structure and on the fact that 
there are functional reasons for differences in structure. In genre pedagogies, it is 
common to work with reading and writing in parallel, for example through the “circle 
model” (e.g. Rose 2005), which is based on mutual deconstruction and construction 
of texts, and that the teacher and students pay attention to various features of the text. 
During such discussions it is natural to discuss what consequences it would have if 
alternative ways of expressing the content were chosen. 

As said, these models are mainly based on written verbal language, which is all 
well and good. Yet if we bear in mind that texts usually consist of a variety of resources 
and not only writing, it is just as important to highlight the interplay between different 
semiotic resources and how to tackle texts from a multimodal perspective (e.g. Tan 
et al. 2012), not least concerning pedagogic texts. 

Also, working with multimodality has potentials to improve students’ text envi- 
sionment (Langer 2011), or text movability (Liberg et al. 2002), that is, to be able 
to approach and relate to texts and their various functions in different ways. So far, 
students’ text movability has mainly been studied in relation to the verbal aspects of 
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written text. However, a Danish small-scale study by Jesper Bremholm (Liberg et al. 
2012) describes how secondary school students show more limited text movability 
in regard to multimodal aspects of texts in science compared to the verbal aspects of 
the same text. 


4.5 Model for Working with Multimodal Texts in Education 


Note: The presentation of the model is based on an article published in the Open 
Access Journal Designs for Learning (Danielsson and Selander 2016!). That article 
gives the foundations of the model and this is done through the same text examples 
as used in the following. However, due to reasons outside of our control, we have not 
been able to obtain a permission to use examples from one book in the following, 
and therefore, we refer to the original article regarding the Chilean examples in the 
following. 

As said previously, from an educational perspective, we believe that “the expert” 
(often the teacher) might need tools to identify potential challenges in multimodal 
texts. The model presented in the following can be such a tool. Furthermore, we 
believe that teachers can use the same model as a basis for classroom discussions 
about multimodal texts with students, who are likely to be “novices” as regards the 
content in focus. 

Table 4.1 presents our model for working with multimodal texts in education. In 
the development of the model we have drawn on social semiotic and multimodal 
perspectives of meaning-making (Bezemer and Kress 2008; Jewitt et al. 2016; Kress 
and van Leeuwen 2006), and specifically so in subject-oriented knowledge domains. 
The model is empirically grounded on research revealing that meta-textual classroom 
discussions generally are scarce in classroom practices, and in particular in relation 
to multimodal aspects of texts. 

Briefly, the model comprises the following aspects of texts: 


The general structure of the text; 

the interplay between the various resources used in the text; 
figurative language; 

values (explicit and implicit). 


Thus, besides focusing on aspects such as the general structure of texts and the 
interplay between different resources, which are central from a multimodal perspec- 
tive, the model includes the use of figurative language (metaphors, analogies, similes, 
see for instance Cameron 2002) as well as values. In the following we give some 
comments on that. 

Research and theory concerning metaphors and analogies is vast (one key volume 
is Lakoff and Johnson 1980). There are several ways of defining these terms and 
it is not always easy to draw a sharp line between them. For the purposes of this 


‘https://www.designsforlearning.nu/articles/10.16993/df1.72/. 
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Table 4.1 Model for working with multimodal texts in education (based on Danielsson and Selander 


2016) 


General structure 


Multimodal text focus 


Thematic orientation and 
sequencing (What is the text 
about and how is the content 
arranged?) 


What does each one of the text 
resources express? (images, 
diagrams, headings, 
introductory paragraphs, etc.) 


Classroom focus 


Focus of the overall structure of 
the text and of the different parts 
of the text and what content they 
provide 


Interaction between text 
resources in the text 


Figurative language 


Proximity and coherence 
between different text resources 


Congruence and coherence 
between concepts, descriptions 
and explanations given through 
different text resources 


Metaphors (incl. analogies and 
similes) in different text 
resources 


Reflection on the interaction 
between different text resources 
and what aspects of the content 
appear central 


Deconstruction of the figurative 
language given through different 
text resources. What is the reach 
of the metaphors and analogies? 
How well do they function for 
the specific content? 


Values 


Explicit 
Implicit (e.g. in metaphors, 
images, perspectives) 


Discussion of aspects such as 
right/wrong, us/the others, 
female/male, etc 


book, however, we draw on the central idea that all definitions include a possibility 
of activating two distinct domains, and in line with, for instance, Cameron (2002), 
our use of metaphor includes both of these types as well as explicitly expressed 
similes (e.g. “my love is like a red red rose”). Since we here focus on multimodal 
texts, we also include other means of representing metaphors apart from linguistic 
expressions. 

Figurative language is a natural part of the disciplinary discourse in many areas 
(e.g. ‘monetary flow’ or ‘magnetic fields’) and is frequently used as a way of under- 
standing and talking about complex structures or processes. Also, such expressions 
function as pedagogical tools, for example to make “visible” what is not possible to 
see directly (like micro-worlds, or rather abstract phenomena as a ‘monetary system’, 
or concepts like ‘democracy’). To some extent, figurative language can function in 
the same way as a visual model. By using it, it is possible to make an inner visu- 
alization that summarizes the main points. At the same time, a metaphor can also 
be a simplification and generalization and it is not always clear how far an analogy 
or metaphor reaches. For example, it might not be obvious what parts or aspects 
of the source domain (e.g. electron shells in the example mentioned in Chap. 3) 
and the target domain (e.g. the atomic structure) are actually similar, and what parts 
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or aspects are not. Especially for the “novice” in the field the reach of metaphors 
might not be obvious. Therefore, even though metaphors can support students in 
their meaning-making, such use has also been noted to be potentially challenging, 
for instance if students take an analogy or metaphor too far or interpret them literally 
(e.g. Haglund 2013; Danielsson et al. 2018). In the case of the atom, the metaphorical 
use of “electronic shells” to explain the ways in which electrons move around the 
nucleus at a certain distance, gives an impression that there are actually shells where 
electrons can be placed. 

Values, both explicit (“you should eat healthy food”) and implicit (“Huowei 
skipped breakfast; therefore he is tired”), can also be expressed through different 
semiotic resources. Since images can be used to convey values in implicit ways, 
multimodal perspectives have previously proven fruitful for such analyses (e.g. Kress 
and van Leeuwen 2006), something which has also been emphasized in relation to 
pedagogic texts (e.g. Unsworth 2007). 

Included in our model are also comments on how teachers and students can work 
stepwise, unfolding and discussing the text, thus supporting meaning-making around 
the content. In such discussions, an overall perspective can be that there is always an 
author behind a text, and that this author has made a number of choices in regard to 
choice of content, as well as what resources to use and how. This way of discussing 
texts, highlighting that there is always an author behind it, is central for the framework 
Questioning the Author mentioned above. That framework, however, is based on 
verbal text only. To discuss texts in explicit ways is important, not least because the 
novice in a field of knowledge cannot be expected to be able to fill in the missing 
parts, or make explicit the implicit links or relations between different pieces of 
information. By highlighting that there is always an author that has made choices, 
can be a way of connecting to the students’ own text production in different school 
subjects. Furthermore, by highlighting authors’ choices, students who might find 
texts challenging can feel confident in actually questioning the author’s choices 
rather than questioning their own capability to make meaning from the text. Such 
discussions can also function as a way of supporting the students’ development of 
critical literacy (e.g. Luke 2000). Altogether, the different aspects of the model 
underpin the development of meta-cognitive skills through an understanding of how 
different resources operate to represent knowledge. 

As mentioned, a number of frameworks for working with texts have been devel- 
oped previously, though generally with a bias towards writing, thus disregarding 
other resources, such as tables, graphs, and diagrams. Examples of exceptions are a 
model presented by Unsworth (2001) and the framework Multimodal Analysis Image 
(Tan et al. 2012). In comparison to those models, our model has a stronger focus on 
subject content and meta-textual classroom discussions. Thus, it has a double benefit 
in that the model supports students’ meaning-making related to subject content while 
at the same time having the potential to support students’ development of multimodal 
literacy.” 


? Multimodal Analysis Image (Tan et al. 2012) has been used in exploratory studies in Singaporean 
schools (Lim et al. 2015). However, rather than focusing on content and multimodality, the main 
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Fig. 4.1 Food digestion, lower secondary school, Science Matters, Lower Secondary (Fong et al. 
2013, pp. 24-27, with permission from Marshall Cavendish Education) 


Our presentation of the model is based on sections on human digestion in two 
textbooks in science, to illustrate the different parts of the model. One book is Singa- 
porean, used for lower secondary school (students aged around 14); the other one is 
Chilean, used for middle-grade students (students aged around 11). Figures 4.1, 4.2, 
4.3 and 4.4 are images taken from different parts of these sections in the Singaporean 
textbook. As mentioned, the images from the Chilean textbook can be found in the 
original article presenting the framework (Danielsson and Selander 2016).* For the 
complete chapters of each of the books, we refer to the original text books. Our point 
here is not to make any kind of comparative cultural analysis. Instead, we want to 
demonstrate that the framework is versatile and applicable across different cultural 
contexts, with a focus on how the orchestration of text resources through different 
semiotic modes has consequences for how to grapple with the text as such and for 
the interpretation of the content. Here we concentrate on paper-based pedagogic 
texts. With minor adaptations, the model can also be used for digital texts involving 
resources such as moving visuals, sound, and hypertexts. We use here examples from 
one subject to illustrate the various parts of the model. In the second part of this book 
we use it to analyze texts from a great variety of subjects. There it will be clear that 
in different texts as well as in different subjects, certain parts of the model might be 
more important than others. 

Our point of departure is how different text resources are used to build up the 
text as a whole, how different text resources interplay and the potential challenges 
regarding the different resources as well as their interplay. Also, we comment on 
the implications for education, suggesting ways of discussing the texts. Such text 


intent in these studies has been to cultivate students’ multimodal and critical literacy, for example 
by letting students analyze and create multimodal texts in relation to advertisements. 


3The original article including all images, can be found here: 
https://www.designsforlearning.nu/articles/10.16993/dfl.72/ 
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What happens to food in the gut? 


Fig. 4.2 Food digestion, lower secondary school, Science Matters, Lower Secondary (Fong et al. 
2013, pp. 28-29, with permission from Marshall Cavendish Education) 


Fig. 4.3 Food digestion, lower secondary school, Science Matters, Lower Secondary (Fong et al. 
2013, pp. 32-33, with permission from Marshall Cavendish Education) 
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Fig. 4.4 Food digestion, lower secondary school, Science Matters, Lower Secondary (Fong et al. 
2013, pp. 38-39, with permission from Marshall Cavendish Education) 


discussions must of course be adapted to the students” age, interests, and previous 
experience of reading and discussing various kinds of texts. 

The model is relatively extensive, but there is no need to always use all parts of the 
model for all texts. Instead, depending on the content area and the purpose for which 
a specific text is used in the educational situation, different aspects of the model can 
be concentrated on. In the second part of the book, we have made a number of close 
readings of texts in different content areas. In relation to those readings, we also 
give some examples of what could be fruitful to focus on in text discussions in the 
classroom. 


4.5.1 General Structure—Setting 


The analysis of the general structure can be connected to the notion of setting in the 
designs for learning framework (Selander et al. 2021). In our model, the analysis of 
the general structure is meant to capture the ways in which a text “invites” its reader 
and calls for certain types of activities by its means of representing the content. Here 
we start off by looking at the thematic orientation, and then the sequencing of the 
text. After that, we go on to examine what the different text resources express in 
regard to content, such as different kinds of illustrations, headings, and text boxes. 
The aim of examining texts in regard to the general structure is to get an overview 
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of the text as regards both layout and content. For meta-textual discussions in the 
classroom, this is a suitable starting point, and even quite young students can be 
involved in such discussions. Things to highlight in the discussions could be what 
content seems to be expected to be found in the text (from the information that, for 
instance, headings and illustrations offer to the reader), what resources “stand out”, 
what roles different types of illustrations seem to play, and if there seems to be an 
expected way to go about reading the text. 

In regard to the thematic orientation of the Singaporean text, the miniature in 
Fig. 4.1 shows the starting point of the chapter “Human digestive system”. The left 
page is the first one of the introductory section headed “Why is the digestive system 
important?”. This heading provides the students with the main concept, “the digestive 
system”, and by expressing the heading as a question, the intention seems to be to 
draw the students into the content by making them curious to find out the answer. 

The chapter as a whole has a structure that evolves from a direct connection to 
students’ everyday lives to a purely scientific approach to the content. It starts off with 
a short text about a boy who has skipped breakfast and therefore is feeling hungry 
and weak. Then a table presents nutrients with comments on their molecular sizes, 
function, and examples of food in which they can be found (see miniature in Fig. 4.1). 
Through this table the reader is taken one step further into the scientific aspects of 
the content. The remaining part of the chapter on the human digestive system mainly 
deals with the digestive system from a scientific point of view, including chemical 
aspects of digestion (e.g. large image in Figs. 4.1, 4.2, and 4.3). 

Even though the chapter takes on a gradually more scientific approach, different 
activities, sometimes labeled 21st Century Skills (Figs. 4.1, 4.2 and 4.3) connect 
the topic to the students’ everyday lives throughout the chapter. Towards the end of 
the chapter, the human digestive system is summarized in a flow chart (Fig. 4.4). It 
is worth noting that it is not until the flow chart at the end of the chapter that the 
terms physical and chemical digestion are defined, even though the terms are used 
throughout the chapter. 

The section on digestion in the Chilean textbook begins with a book spread with 
a series of pictures resembling a comic strip with some writing below (for images, 
see Danielsson and Selander 2016). The following bookspread consists of a running 
text in the upper part of the bookspread, with a series of images below that. In regard 
to the thematic orientation of the Chilean text, it also starts out with a a heading 
formulated as a question(“¿Cómo funciona nuestro cuerpo?”, Eng. “How does our 
body function?”), inviting the reader to learn more. Here, the starting point is the 
series of pictures over the double-page spread in which two children decide to make 
an imaginative journey through “the body”. The journey goes from the stomach to 
the heart via the blood and after a detour to the lungs back to the blood circulation, it 
finally ends with the children floating in urine in a bladder. During the journey, the 
two children note things such as nutrients and oxygen entering into the cells of the 
organs. 

Underneath this, under the heading ‘I observe and respond’ (Spa. “Observo y 
respondo”), students are given questions that are supposed to make them read the 
comic strip more closely, for example by questions about the different body organs 
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that the children encounter. On the left page of the double-page spread, a number 
of questions invite the reader to describe, for instance, the essential functions of 
different body systems. However, there is no explicit information in the comic strip 
that gives specific information about these systems. 

The following pages in the Chilean book go deeper into the subject-specific 
content. Here a series of pictures at the lower part of the bookspread gives grad- 
ually more abstract and detailed representations of parts of the intestines, starting 
with a photo of a child eating an apple, ending with an image of a cell which is said 
to be one of the many cells building up the intestines. 

In the comic strip as well as in the writing accompanying the illustrations in 
the following book spread, scientific terminology connected to the topic is used 
frequently. The following section deals with the physical and chemical aspects of 
the digestive system. Here labels give the scientific concepts connected to the human 
digestive system presented in an image, similar to the Singaporean book spread in 
Fig. 4.4. An explanatory text describes the kind of processes that go on in the various 
parts of the system shown in that image. 

As we note from the analysis of the thematic structure in the books, there are 
similarities between the texts. Both of them invite the reader to the content by using 
questions as headings, and they both use a connection to the students’ everyday lives 
as a Starting point. 

Sequencing deals with text structure at a general level, for example to what extent 
the information structure invites the reader to read the different parts of the text in a 
certain order. From a brief look at the book spreads in Figs. 4.1, 4.2, 4.3 and 4.4, and 
the two bookspreads from the Chilean book (see Danielsson and Selander 2016), it 
can be noted that it is only the second bookspread in the Chilean book that has a tradi- 
tional structure, with a body text clearly separated from the illustrations. In the other 
texts, different textual and visual modes are intertwined. Another way of sequencing 
is found in the book spread from the Singaporean textbook in Fig. 4.1, where the left 
page is dominated by the image depicting a cell. Such a prominent image, in both 
relative size and central placement, draws the attention to that area of the page (see 
Kress and van Leeuwen 2006, for a discussion of salience). At the same time, the 
numbers | and 2 imply a reading order starting at 1. This type of visualization, with 
the prominent image of the cell, including a zoom-in on a cell membrane alongside 
numbered text sections and arrows, is common in pedagogic texts. Such visualiza- 
tions imply a reading order following the arrows, where the reader is supposed to 
juggle between writing and visualizations, using the arrows to follow a process. In 
this case, however, the numbered arrows have partly different functions, with two 
arrows (one crossed over, being a “dead end”) pointing in divergent directions from 
the verbal text numbered “1”. The arrow that is crossed over functions as a notice that 
big molecules cannot pass into the cells, and therefore (which is not stated explicitly) 
we need the digestive system to make big molecules smaller. Thus, drawing attention 
to the general overview and the ways in which the reader is “invited” into the text 
can function as a form of guidance for the students, at the same time as possible 
challenges (such as the use of arrows for different purposes in the visualization) can 
be highlighted in such discussions. 
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With regard to what the different semiotic resources are used for, we concentrate 
mainly on the book spread shown in the large image in Fig. 4.1. This spread contains 
images of various kinds (abstract images, like the human cell and the molecules, 
as well as more realistic images, such as the black and white photo of a scientist), 
Writing in different forms, for instance body text, headings of various levels, and 
words integrated in visualizations. 

The image on the left page is used for showing aspects such as wholes and parts 
of the cell and for giving a more concrete image of abstract content, such as simple 
and complex molecules. At the same time it gives a kind of time sequencing (large 
molecules breaking down to smaller molecules) and processes (simple molecule 
passing through the cell membrane). This explanatory use of the image can be 
compared to the more illustrative images on the previous spread in the textbook 
(miniature in Fig. 4.1), with the tired boy resting his head on a desk, and food on 
plates, or the schematic image on the left page of the following book spread, shown 
in Fig. 4.2, labeling different parts of the upper digestive system. 

Also, various kinds of graphic devices are used in the book spread shown in 
Fig. 4.1, for example text boxes and bolded words in the body text. The use of text 
boxes of various kinds is common in textbooks. In the Singaporean book, text boxes 
are used to mark key ideas, using an explicit heading. Throughout the book, recurrent 
activities intended for the students to go beyond the immediate “facts” in the book to 
connect them to real-life situations are marked “Think and Explore”, with an explicit 
connection to “21st Century Skills”. The Chilean textbook also provides the reader 
with a number of key concepts in a text box shown in the second book spread (see 
Danielsson and Selander 2016): in this case different types of body tissues (Spa. 
tejidos), which are given in bold text. However, this information is given under the 
heading ‘Did you know?’ (Spa. ¿Sabias Que?). Such headings are quite frequently 
used for texts intended to raise the curiosity of the reader, but perhaps giving less 
important information. 

Since there are no “rules” as to how to use different graphic devices in a text, there 
are good reasons to discuss and make explicit how different resources are used in a 
specific text when introducing it in an educational context. 


4.5.2 Interaction Between Text Resources in the Text 


An important aspect of multimodal texts in education is the relationship between 
the different text resources on the page (or equivalent), and the different ways these 
resources are used for expressing various aspects of the content. When examining 
the interaction between text resources, we examine to what extent the different 
resources give the same, overlapping, or different/supplementing information (also 
see Unsworth 2007). When different resources supplement each other, they can some- 
times appear to give partly contradictory information, at least for the novice in the 
field. One such example is when a concept like the atom is presented as a static 
phenomenon consisting of various particles in an image, while at the same time 
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other resources like verbal text or gestures (in the classroom practice, or in a video) 
focus on the dynamic aspects of the atom, with electrons swirling around a nucleus 
(e.g. Danielsson 2016). 

Both students and teachers tend to view images of various kinds as a way of 
facilitating the reading of a complex text (Danielsson 2011). Images can of course 
be used to visualize or simplify a complex phenomenon or reasoning. However, they 
might also add new, complex—or contradictory—information. Therefore, images 
too can be challenging for the interpretation of the text. 

When creating multimodal texts, one always need to decide whether some content 
should be given in images (and why). Besides that, a number of other choices have 
to be made as to how to depict the content in image (photo, drawing, graph, etc.) 
and what level of abstraction that would be suitable for the content in relation to, for 
example, the intended reader. Here we can make interesting comparisons between the 
series of pictures in Fig. 4.3 from the Singaporean textbook, and the images shown 
in the second bookspread in the Chilean book (see Danielsson and Selander 2016), 
which to some extent are used for parallel content, such as the fact that the intestines 
play an important role for the digestion. The Chilean textbook is intended for slightly 
younger students. In this case, the series of pictures starts with an everyday, concrete 
photograph (a boy eating an apple) and moves towards more abstract representations 
(a drawn image of the cell). The Singaporean textbook, on the other hand, shows a 
less concrete image of a body (presented in a kind of X-ray fashion), and this image 
is supplemented with a zoom-in which gives a schematic image of the walls in the 
small intestine and a blood vessel, where “small molecules can pass”. This zoom-in, 
in turn, is supplemented with a schematic image of the cells. The series of pictures 
with gradually more abstract content in the Chilean textbook could be an attempt to 
“lead the reader by the hand” towards the representation of the cell, which is quite 
distant from an idea of “body parts”. The Singaporean textbook, on the other hand, 
makes higher demands of the readers’ capacity for abstract understanding right from 
the start. 

As regards images presenting content that is also mentioned in the body text, 
various ways of connecting, for instance, terminology to an image can be used to 
facilitate meaning-making for the reader. One such way is to ensure spatial proximity 
between a term used and the equivalent part of picture, that is, to give verbal comments 
on the image located relatively close to it. In Fig. 4.5, presenting an image from 
a popular science magazine, proximity is further enhanced by the use of colors. 
Arrows or lines between verbal text and image are also commonly used when labeling 
important parts of an image (e.g. upper right of large spread in Fig. 4.1). 

Another important aspect concerns consistency in terminology use between 
different representations. The use of terminology shown on the page in Fig. 4.1 
is consistent throughout the page to the left in the large spread, with specific words 
like “cell membrane” re-occurring in the visualization as well as in the verbal text 
integrated in the visualization and in the text presented in other parts of the page. 
However, a closer inspection of “the cell” in Fig. 4.1 and 4.3 reveals that this concept 
is depicted quite differently depending on what aspects of the cell are highlighted in 
the different sections. Such a variation is something which might need to be pointed 
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Fig. 4.5 Enhancing proximity by use of color: BBC Focus Magazine (Ridgway/BBC Knowledge 
2013, p. 71, with permission from BBC Science Focus Magazine) 


out in the educational context, and discussions about the different choices can support 
the students in relation to both subject contentand multimodal literacy. 


4.5.3 Figurative Language 


As mentioned, there are several reasons to focus on figurative language such as 
metaphors in relation to multimodality. Firstly, a metaphor in itself can often be 
seen as a multimodal ensemble (e.g. Jewitt 2014). An example is the use of the 
metaphor “monetary flow” to talk about economic systems. Here the words can lead 
to a mental visualization of a flow. Secondly, metaphors can be expressed through 
different semiotic modes in a text (writing, image, etc.). Also, some metaphors are 
integrated in the content of a knowledge domain (see below) and, consequently, 
central for meaning-making in that area. 

In the following, we discuss figurative language in writing as well as in images, 
since images in pedagogic texts can be based on metaphors. One such example is the 
welfare system of a society, where an image could be in the form of water flowing 
in pipes with the different parts interconnected as in a technical system (cf. “flow of 
money”). 

As mentioned, the use of metaphors can also be challenging, for instance in 
regard to the reach of a metaphor. Also, as regards second language learners, the 
use of uncommented figurative language can be potentially challenging. This is 
particularly true when everyday expressions are used (which is often the case with 
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metaphors, for instance “muscular bag”). For a student who knows the everyday 
expression, the figurative use can be misleading. Therefore, an important role of the 
teacher is to “unwrap” the figurative language and to help students to focus on the 
adequate aspects of the content expressed metaphorically. 

In the two texts we have chosen, we can find some interesting examples of figu- 
rative language. In the Singaporean text, we are told that glands produce “special 
juices”, and that there is “pancreatic juice”, “intestinal juice” as well as “gastric juice” 
(Figs. 4.2, 4.3 and 4.4). The concept of “juice” is not explained, neither in relation to 
the everyday language of juices nor in terms of the different kinds of juices within 
the different parts of the body. 

Other metaphoric expressions are “muscular tube”, “muscular bag” and “wall”, 
or “how food travels” (Fig. 4.2, our italics) through the body. Also, we learn that 
“hydrochloric acid [...] helps the protease to work” and that “Long muscular tubes 
contracts and relaxes, which mixes intestinal juice and pancreatic juice with food, 
to help digestion” (our italics). Apart from being metaphoric expressions, food and 
muscles also become “humanized” with intentions in these examples.* 

We can also notice more general concepts like “systems” (for example “systems 
that form the organism” in Fig. 4.4) and “functions”, which have connotations of a 
machine-like mechanism, where every part has been made in relation to every other 
part to serve an overarching whole. In the Chilean book (see Danielsson and Selander 
2016) it is said that systems of digestion, circulation or nerves “together form the 
organism”. In the Singaporean book, the flow chart of the digestive system in Fig. 4.4 
further enhances the notion of a system in a type of visualization. 

An interesting kind of analogy is found in the Singaporean text (Fig. 4.3, bottom, 
left). Here the students are supposed to perform an experiment with water, cooking 
oil, and washing-up liquid as an analogy for the digestion process when fats break 
down in the body. In the activity, it is explicitly mentioned that the cooking oil “plays 
the role of” (i.e. functions analogous to) the fats in the food, while the washing-up 
liquid plays the role of the bile. However, the analogous role (if any) of the water is 
implicit. 

Discussions about the use of figurative language in a text, including “unwrapping” 
the metaphors and their reach can function as excellent opportunities for in-depth 
discussions of content matters (see also Danielsson et al. 2018). 


4.5.4 Values 


Value statements are a part of human communication. Sometimes values are openly 
declared, and we can simply agree or disagree with them. But equally important—or 


4Such anthropomorohic metaphors have been noted in previous research in science classrooms (see 
for instance Tibell and Rundgren 2010, Danielsson et al. 2018). In a recent study, Pettersson and 
colleagues (2020) noted a great deal of anthropomorpisms when students were asked to describe 
the digestive process in a national test. 
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perhaps more important—are the more implicit or tacit values in a text. In these cases 
we need to scrutinize the ways in which the information is presented, and what kind 
of information is left aside. 

In the two texts we have analyzed here, explicit articulations of values are not 
frequently used. But one example can be found in Fig. 4.3 (right) under the sub- 
heading “Think and explore”. The text states that “Some teenagers become obsessed 
with losing weight [...]” and then follows a warning concerning possible dangerous 
consequences such as eating disorders (anorexia and bulimia), which might cause 
“life-threatening conditions” such as “muscle weakness, kidney damage, heart failure 
and even death if left untreated”. 

Implicit values are of course more difficult to detect, since they very much depend 
on the perspective from which the text is interpreted. Here we would like to point out 
some interesting examples. Curriculum documents often state that historical links 
should be made in different content areas, though exactly what historical links are 
supposed to be offered the students might not always be stated. In the Singaporean 
text a historical connection is made through some information about Dr. William 
Beaumant, here highlighting his efforts and curiosity, while the fact that he used a live 
person for his experiments is not problematized (Fig. 4.1, right). Another example 
of implicit values is the conceptual hierarchy (Fig. 4.4, right), which shows how 
different concepts are interrelated and, implicitly, how scientific conceptualizations 
should be organized in more or less general terms. In this case, the text shows us 
what kinds of representations are valued in scientific work. The hierarchy is also a 
way of pointing out the role of scientific concepts in relation to the everyday use of 
different terms. 

It is important for students to become aware of the ways in which values can be 
more or less hidden, in order to prevent them from being manipulated by texts and 
for them to develop their critical literacy (Luke 2000). Also, by highlighting implicit 
and explicit values in texts, interesting discussions connecting to the students’ own 
life experiences can be made possible. Even though values can be commented on in 
relation to any text, this might not be the most important aspect of the two texts in 
focus here. In the second part of this book, however, we have analysed texts taken 
from history and religion (Sects. 8.5 and 8.6), which are subjects that are particularly 
interesting concerning values. 


4.6 Summary 


In this chapter we have focused on educational perspectives of multimodal texts, and 
we have also presented a model for working with such texts. 

The fact that different multimodal texts and different text genres or texts in 
different subjects can be challenging for the reader in different ways must lead to an 
extended view of the development of reading and writing. From such an extended 
perspective, “learning to read and write” means developing different repertoires 
regarding different genres as well as texts in different subject-specific contexts. The 
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way that “the expert” can interpret texts is different from that of the “novice”. For 
example, a person who possesses great knowledge in a field will fill in potential gaps 
in a text, in ways that the novice cannot do. Therefore, a teacher with a profound 
understanding of the content area might not always realize the potential challenges 
regarding the interplay between textual resources such as writing and various kinds 
of illustrations. Hence the expert might need tools to make visible the challenges 
implied in a text, in order to support novices in their encounters with texts. 

A number of methods for working with reading have been developed previously, 
though most of them focus on verbal language. Our model focuses on the multimodal 
text and how to work with such texts in education. In the last section of the chapter, we 
have presented our model for work with multimodal texts, alongside a presentation of 
the different aspects of the model by analyzing two texts, focusing on (1) the general 
structure of the text, the thematic orientation and sequencing, and the different text 
resources used; (2) the interaction between the various parts of the text, such as visual 
proximity and congruence as regards the contents of the text; (3) figurative language, 
such as analogies and metaphors; and (4) explicit or implicit values. 
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Chapter 5 A) 
Cultivating Students’ Text Creation checi; 


Work with multimodal texts as described above is also relevant for students’ own 
text production. Furthermore, parallel to the importance of knowledge about multi- 
modality in regard to students’ meaning-making with pedagogic texts, there is a need 
for an enhanced knowledge of multimodal writing practices, and how to support 
students in their writing development. Also, teachers need ways to assess students’ 
multimodal products in fair and relevant ways. Such questions have been discussed 
within the field of New Literacy Studies (e.g. Bearne 2009; Cope and Kalantzis 
2000), though it appears as if teachers feel that they lack the tools for working with 
multimodality (Bearne 2009; Engblom 2011). Nurit Peled-Elhanan (2015) noted that 
teachers tended to see children’s use of images as “lack of capability, a transitory 
phase or waste of time” and she concluded that “[a]lthough most of the texts children 
read at school are multimodal they have no license to develop multimodality in their 
own texts” (Peled-Elhanan 2015, p. 275). Based on discussions in the field, we will 
give an example of how to use our model in relation to the students” work with their 
own multimodal texts. 


5.1 Writing Practices in Schools 


In school, students are involved in a variety of writing practices. They might have 
learned that they can underline text or use other ways of highlighting important 
text passages, they make their own notes and drawings, or they take photos of the 
teachers’ written expositions. This is a kind of private writing, used for the student’s 
own thinking processes and seldom intended for anyone else than the student herself. 

The log book is another form of writing which is something in between private 
reflections and the more official writing practices of the school as an institution. In 
the log book, private experiences from the teaching, perhaps including reflections on 
social aspects of the learning situation can be mixed with notes of direct relevance 
to the subject content. 
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Examples of more or less public and visible writing activities are experimental 
reports or other kinds of written presentations connected to the subject content 
(these reports can be presented through PowerPoint presentations, Internet pages, 
wall posters, etc.), essays, or compositions. Such writing activities are supposed to 
focus mainly on what is highlighted in relation to the subject content. Genres such as 
the experimental report are fairly standardized, while the essay gives room for more 
variation and personal comments. 

One crucial question is what kind of writing best supports the understanding 
of subject content. It could be claimed that all kinds of writing are beneficial for 
students’ learning, no matter what it deals with or what kind of writing is involved. 
An argument against such a position is that too much emphasis on, for example, 
log books, can lead to a situation with many unstructured texts, something which 
can be counterproductive, considering the expectations on other texts, such as written 
reports. Therefore, teachers need to be explicit about the didactic function of the texts 
when students are asked to produce texts: is the text supposed to be a thinking tool, or 
will it be used for communication with others about the subject? For the latter type, 
explicit instruction and joint text construction (cf. Rose and Martin 2012) can harness 
the students’ writing process and their development of metatextual competencies. 
When working with texts intended to communicate with others, the natural starting 
point is the function of the text: why should it be produced, and who is supposed to 
make meaning from it? With such a starting point, a number of choices will need 
to be made concerning what aspects of the content to include, and how to express 
different aspects of the content (cf. the central questions of why, for whom, what, 
and how?). Also, anyone who produces a text can decide whether to “play it safe” 
and adhere to genre expectations, or whether to challenge them.! From a multimodal 
point of view, it is then important not “only” to focus on linguistic choices, as is often 
the case in, for instance, genre pedagogy where joint text construction is a central 
didactic tool. Instead, choices in regard to other modes should be equally focused. A 
central idea, then, is that every choice concerning how is always tightly connected 
to why and what, where why should always be the starting point (see Bergh Nestlog 
and Danielsson 2020). 


5.2 Linear Writing and Multimodal Composition 


In text traditions based on written language, you write, for instance, from left to 
right or from right to left (that is, in cultures based on such orthographies, such as 
English, Spanish, Hebrew, Arabic). One consequence of the written language bias is 
that the writing structures have evolved based on words, clauses, and text sections, 
with different kinds of punctuation or emphases. Writing in such cultures follows a 


I This can be connected to and idea of a”double dialogue” (cf. Evensen 2002, who draws on Bakhtin), 
namely the idea that when producing a text we are interacting both with the one(s) that the text is 
intended for, and with conventions in regard to similar texts (genre expectations). 
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kind of time line characterized by “first... and then” or thematic structures where the 
point of departure (the theme, or what is already known) in a clause is mentioned 
first, while new information is placed at the end of a clause (Halliday and Matthiessen 
2014). If you state, for instance, that “Sarah and Kim have just bought a new house” 
you presume that Sarah and Kim are known to the reader or listener, while the fact 
that they have bought a new house is presumed to be new information. If we instead 
say that “The house was sold by Sarah and Kim”, we then have another perspective. 
Here it is presumed that the house is already known to the reader or listener, and 
perhaps also that fact that this house is about to be sold. 

In linear writing, a number of significant features have been developed in relation 
to different text types or genres: the structure of the short news item is different 
from that of the editorial, an explanation is different from a descriptive or narrative 
text, and so on (e.g. Martin and Rose 2006). Such recurring structures are taken 
into account in genre pedagogy. As opposed to linear writing, texts built on visual 
elements have a different structure, based on spatial orientation, where elements with 
a central placement or which are big in size or taken close-up receive greater weight 
than elements placed on the periphery, or viewed from a distance, where the choice of 
a schematized representation or a representation of higher fidelity can signal different 
levels of scientificness (see Kress and van Leeuwen 2006). 

As we discussed Chap. 2, different kinds of resources or semiotic modes are 
better apt for different kinds of communication or ways of conveying knowledge, 
as is illustrated in an example reported by Kress (2010). After a visit to the British 
Museum in London, pupils were asked to use images and writing to make a text about 
their experiences. One boy gave a relatively detailed, temporally based, narrative 
about practical things like the fact that first they left their clothes in the cloakroom 
and later picked them up, that they used the elevator to get to the exhibition, and 
that they went home by tube and train. He used one sentence to report from the 
exhibition: “Then I went to see the mummies and all that stuff’. Thus, in writing, 
he described a lot of details in relation to the visit, but in his drawing he chose to 
portray the mummies. In this case, the image was used to focus on something the 
boy found quite spectacular, perhaps because the mummy trigged his engagement 
and fantasy in a way that he could not express in the narrative (Kress 2010, also see 
van Leeuwen 2000). 

However, regardless of the actual reason why this boy chose to draw the mummy, 
in a classroom discussion, the teacher could discuss different things; for example 
what the boy had chosen to present through the different resources, what aspects of 
the mummy he found particularly interesting, or what details were especially salient. 
Here, the image, rather than the written narrative, would be the natural basis for 
assessing the boy’s knowledge about mummies, and consequently, his knowledge of 
ancient history. 
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5.3 Towards a New Understanding of Writing 


In the multimodal text, the logic of writing is usually mixed with the spatial logic 
of the image. Different types of information are given through different resources. 
Hence, today, writing is a process of composing a text, combining different types of 
text resources to form a coherent text. Such a composing process will be different 
from composing a text entirely based on more traditional writing. Therefore, when 
composing a multimodal text, a number of choices are important. For example, it is 
important to decide what semiotic mode to use to express what aspects of the content, 
or if the content given through different text resources should complement, or overlap, 
each other. Apart from that, choices have to be made in relation to typeface, size, 
and colors in images or other visual resources, and where to place them in relation 
to each other. For anyone interested in reading more about multimodal composition, 
we refer to literature that can enhance the semiotic analysis (e.g. Kress 2010; van 
Leeuwen 2005, 2011). 

In a text based on writing, we normally presume that the text is formulated by the 
author and if parts are taken from other sources, quotation marks should be used and 
the source needs to be stated. However, this has not always been the case. When the 
printing of books was developed, books could consist of compilations of a variety of 
texts, and thus, the role of the author was more of a “text compiler” than an author in 
a modern sense (Swann 2001/2010). A similar approach is today’s ways of handling 
digital texts, where compilations and combinations of texts are mixed with texts 
written by a sole author (for example the case of fan fiction which was mentioned 
above). Therefore, for teachers it is important to ponder over how to approach the 
use of different sources in students’ texts. 

In Sect. 5.1 above, we mentioned that in the text traditions of (Western) schools, 
students in lower grades have been encouraged to combine writing with image. For 
the really young children, image is the main source of meaning, while the verbal 
text can be seen more as a backup or illustration (not seldom with a grown-up or 
older child who can act as the writer for a couple of words or sentences). Later we 
can note a shift, and in school the pattern is the reverse, with the function of the 
image being more of an illustration to the writing. In lower grades, this tradition is 
strong regarding both narrative and more expository texts. In higher grades, the image 
used as an illustration is of less and less importance. However, in natural sciences 
or the social sciences, for example, writing is often combined with graphs, tables, 
diagrams, or images that show various parts of experimental work. Thus, for students 
to create multimodal texts is by no means something new. What is new, though, is 
the availability of digital media in schools, which makes it possible to gain access 
to an infinite amount of data, and to communicate through and produce multimodal 
digital texts. 

The digital world offers, apart from written texts, an enormous amount of images 
or film clips which are freely available for download. It is fairly easy to use images, 
and to combine image with writing, and to give information in images or series 
of images as well as through writing. However, it is not always straightforward to 
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get information about where an image was originally created or by whom. For the 
teacher, it is somewhere along this line that the students” understanding of content 
arise. These challenges can be dealt with in different ways. In some schools where 
they have chosen to invest in one-to-one projects (one computer or tablet for each 
student), students are encouraged to work with subject content through digital tech- 
nology, while test situations are traditional, based on pen and paper (Åkerfeldt et al. 
2013). A question that arises in relation to such practices is whether they really imply 
an understanding of the students” knowledge, or if this way of working only rewards 
a type of knowledge and representations of knowledge that is best shown through 
verbals (writing or perhaps speech). These kinds of questions are even more promi- 
nent when the next generation of digital projects become more common in schools, 
such as in one-to-many-projects, where the students themselves can choose what 
digital resources to work with. 

Hence, the multimodal text world offers both new possibilities for communication, 
and new challenges regarding the texts produced. Texts are redesigned and composed 
in new ways, and one more complicating factor is that texts that are produced in school 
do not stay just there. An increasing amount of texts are made available on the Internet. 
Thus, there could be one culture in the school in relation to school assessement, and 
another one outside of school, based on the student’s individual interests. Examples 
are Internet communities for fan fiction or sites like www.poetry.com where one of the 
incentives for publishing appears to be the—often very positive—assessment from 
other members of the community. This in turn implies that new criteria for cultures 
of assessment need to be developed, based on the communicative situation in which 
the text is created and used (cf. Kalantzis and Cope 2000; Kress and Selander 2012). 


5.4 Supporting Students’ Work with Multimodal Texts 


With minor modifications, our model for multimodal text analysis presented above 
can be used for supporting students’ text creation from a multimodal perspec- 
tive, both paper-based texts and those created through digital media. To support 
students’ composing of such texts entails supporting both their content knowledge 
and their abilities regarding the use of different resources for meaning-making and 
communication. 

We think that some aspects are especially central: the general structure, choice 
of semiotic mode in relation to the content, and the interplay between different text 
resources. These aspects are connected to General structure and Interaction between 
parts of the text in our model. These different aspects of the text must here be related 
to both the function of the text, including the intended reader, and the content. A text 
created mainly for the teacher, with the intention to show one’s understanding of a 
specific content in relation to the curriculum, or other explicit objectives of a course, 
has to meet other demands than a text created with, for example a fellow student as 
the intended reader. 
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Our model makes it possible to go deeply into, among other things, different 
aspects of multimodal texts presented by Eve Bearne (2009). Her model for assesse- 
ment includes the choice of semiotic mode in relation to text function, intended 
reader, text structure, and the use of different (technical) resources for communicative 
effects. 

Another aspect, according to Bearne, is connected to an overall meta-reflective 
ability in relation to texts and multimodality. In this case it concerns the students’ 
ability to reflect upon and enhance their own competence in handling multimodal 
texts. We claim that conscious work with text based on our model can contribute to 
the development of students’ meta-reflective abilities. 

In the following discussion of the development of multimodal abilities, we proceed 
from the various parts of our model, using a student text (Fig. 5.1) as an example. 
This student text is taken from a book on text work by Liberg and colleagues (Liberg 
et al. 2010, p. 16) and it was created in relation to experimental activities in a science 
classroom. Liberg and colleagues do not comment on the student text as such; instead 
it is used as an example of a text created in science classrooms. We also relate our 
discussion to the student text from grade 2 which is shown in Fig. 3.2, Sect. 3.3, 
above. 

When discussing student texts, it is important to consider the situation and the 
conditions prevailing in relation to the production of the text. If students in a chemistry 
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The task: what happens when an 
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Hypothesis: I thought that there 
would be smoke and big bubbles. 


We need: A test tube, A test tube 
MÅ A \ > « ean atect with side tube, a funnel, dish, tripod, 
Uppartiens Vad hender nat tn Syla veageral . A 
Med w agnesiow i sleeve, clip, matches, protective 
glasses, dilute solutions of 
at hydrochloric acid, sulfuric acid, 


nitric acid. 


Hy pote zen i 


tomma 


ù behöver: 
Trah y 


Result: I thought that the 
experiment was fun and that it went 
well. 


Conclusion: The reaction between 
Slotsatg: ka ehonen mellan vate och Ivtter hydrogen and the oxygen in the air 
Syre of mycket Explosive is very explosive 


Fig. 5.1 Experimental report from a science classroom in secondary school, written by hand (Liberg 
et al. 2010, p. 16, with permission from the authors) 
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classroom are told to write an experimental report without having worked with such 
texts before, one cannot expect all students to be able to handle the different genre- 
specific requirements that could be expected in this kind of text, neither in relation 
to text structure and word choice, nor regarding the use of other text resources such 
as images, symbols, etc. One example from the text in Fig. 5.1 is the text under 
the heading “Result” (Sw. Resultat) where the student has written “I think that the 
experiment was fun, and I think that it all went well”. This is an adequate way 
of formulating the results, at least for students who might be familiar with log- 
book writing with self-assessment and reflections about their own achievements. Yet 
this comment says nothing about the understanding of the results in the experiment 
regarding chemical reactions and so forth. 

We have only limited knowledge about the context in which the text in Fig. 5.1 
was produced. In regard to the text in Fig. 3.2, however, we know that the text was 
written after an excursion to the seaside. When the students came back to school, 
they were asked to write a text about the excursion, and as a kind of inspiration, the 
teacher had brought some examples of animals that could be found by the sea, and 
put them into an aquarium. We do not know, however, anything about the extent to 
which the students in this class had focused on the ways in which writing and images 
can be combined, but from Sjghelle’s (2013) description, our impression is that this 
was not the case. 

Thus, regarding the experimental report, we do not know much about the situation 
in which the text was produced, but from our general knowledge about teaching in 
science classrooms we can mention a couple of relevant aspects to have in mind in the 
following. First, science classrooms are highly multimodal learning environments, 
containing subject-specific artifacts such as three-dimensional models of atoms and 
molecules, charts of the system of elements which combine writing with symbolic 
language, and so on. (e.g. Lemke 1998). Furthermore—as in the situation in which 
this student text has been produced—experimental work is quite frequent, and during 
such activities experimental equipment, such as Bunsen burners, test tubes, tripods, 
tongs, and evaporation bowls are used. Also, the experiment is often preceded— 
or succeeded—by a theoretical exposition through writing, teacher talk, gestures, 
blackboard notes, and more (e.g. Danielsson 2016). 

Experimental reports written in schools have been described repeatedly (e.g. 
Knain 2005; Kress 2003) and in many science classrooms students receive some 
kind of text model to use when writing such texts, at least proceeding from a number 
of more or less logical headings. Examples are Hypothesis, Procedure, Equipment, 
and Conclusions. If we look at the text in Fig. 5.1, it seems as if this student uses 
such headings, and we can assume that the teacher has provided the students with 
them. It is also common for students to be expected to combine writing with images 
in their reports, for instance to visualize how the experiment was performed. 

From a multimodal perspective, it is relevant to take a closer look at the choices 
regarding what to present through what semiotic mode. This is specifically interesting 
in regard to how subject content is handled through the different modes, and how 
the text as a whole reflects the understanding of content. The shift in modes from 
the hands-on activity (the actual experiment) to a written report is an example of 
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transduction (Kress 2003). This means a transformation of content from one semiotic 
mode to another (in this case from a hands-on activity to image or writing; in other 
cases transduction could involve spoken verbals and writing, or images and writing). 
When, on the other hand, the shift is made within the same semiotic mode, for 
example when students are expected to “use their own words” when writing a new 
text, for example from a text on the Internet, this is an instance of transformation (e.g. 
Kress 2003). These types of shifts can be quite challenging for students, and anyone 
who creates a text needs to be conscious of the different affordances (or possibilities 
and limitations) of modes to be able to make well-balanced choices. 


5.4.1 General Structure and Setting 


Regarding the choice of semiotic mode and content, the author of a text has to make 
a number of choices about what content is best expressed through what semiotic 
mode or resource. These choices are made in regard to the available modes, which in 
turn depends on factors such as whether the text is paper-based or digital, but also in 
relation to how well you can handle the medium or resource in question (in Sect. 3.2, 
“What semiotic modes ‘count’’”’?, we described how a young student made a spelling 
mistake to make an image appear in the right place of the page). 

The examination of choices of semiotic modes is connected to the affordance, or 
meaning potential, of resources (see Sect. 3.3, above) and to the choice that may be 
best in relation to the function of the text and the (intended) reader. 

In the student text presented in Fig. 3.2 (Sect. 3.3), a primary school student chose 
to draw a crab, but he used words to report that it moved sideways (furthermore, 
through his words he made a general statement: “crabs move sideways”). These are 
well-balanced choices in relation to the meaning potential of semiotic modes. Also, 
the teacher has good reason to utilize this, perhaps implicit, knowledge in future text 
discussions on the choice of semiotic mode (expressed in other words, of course) with 
this student. Here we want to comment on the fact that the student actually colored 
the crab red, which cannot be shown in the black-and-white reprint. It is most likely 
that the crab that they found during the excursion was gray/black rather than red. The 
fact that the boy colored it red—which is the color of a cooked crab—can perhaps be 
explained by the fact that crabs in children’s books and animated films are usually 
red, or the boy might have eaten crab at home. It could also be the case that the 
crab that he depicted was actually red. According to the description of the context 
in which the text was created, the teacher had put a number of things in an aquarium 
as a kind of inspiration. It might be that this was a red toy. 

When writing an experimental report, there are also reasons to combine image 
and words. Images can illustrate what kind of equipment was used, or how an onion 
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that was used for dissection looks in a microscope (e.g. Kress 2003).? Hypothesis, 
results, and the procedure around the experiment can then be expressed in words. 

In the experimental report shown in Fig. 5.1, the student has used a variety of sign 
systems: chemical symbols, a drawn illustration also including figures and words (in 
a caption), writing in the form of body text which has been organized in a kind of 
bullet structure. 

If we look at the text in Fig. 5.1 in regard to general structure and sequencing, 
the text appears to be in line with the experimental report as a school genre. The 
combinations of sign systems (chemical symbols, image, writing) is typical of the 
genre, and the overall impression is that this is a fairly well-structured text. 

A next step in the process is to take a closer look at what is offered through the 
different visual resources, or what content is given through the resources and how 
they are used. On the top of the text, the student has used chemical symbols to write a 
chemical equation. The use of such symbols is typical of the disciplinary discourse, 
where chemical symbols are used to express how substances and different compounds 
react, and what new subjects or compounds are formed during the reaction process. 

Below the chemical equation, the student has drawn an image of the equipment 
that was used during the experiment, and how the different parts were put together. 
Here we can see a tripod with something that looks like a test tube (marked “1”’) 
which is fixed to the pipe through a clip. Beside the tripod is a bowl (marked “2”). 
Between the test tube and the bowl is some kind of pipe. The drawing, too, gives the 
impression of being in line with the genre of experimental report. The writing has 
been sequenced in sections according to (presumably) given headings, such as “The 
task” (Sw. uppgiften), “Hypothesis” (Sw. hypotes), and “Conclusion” (Sw. slutsats), 
which of course is in line with what can be expected, given the genre. 

As a whole, the examination of the general structure of this student’s experimental 
report, gives the impression that the student has created a text that is in line with 
what can be expected of the genre in a general sense, for example through the use of 
different semiotic modes such as chemical symbols, image, and writing. The writing 
is also presented in line with genre conventions, with headings, text in bullets, and a 
caption below the drawing. 


5.4.2 Interaction Between Text Resources 


Apart from examining the interaction between the different textual resources, we 
have elected to include the choice of resource in relation to subject content and 
intended reader. Such considerations could also have been remarked on under the 
previous heading, general structure, where we commented on what content different 
text resources offer the reader. 


?Kress (2003, pp. 109 ff) describes the ways in which students’ visual representations of the onion 
can be related to the ways in which the teacher had talked about the cells forming patterns, similar 
to a brick wall. 
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Among other things, we here look at the coherence between content given through 
the different resources and how they complement or overlap each other. If we return to 
the student text in Fig. 3.2, we can note that, apart from describing different aspects 
of the crab through image, and words, the student has used images and words to 
describe the bad weather during the excursion (a drawing of a sad sun combined 
with the words “grumpy wind and waves” (No. “sur vin og bølge”); here the student 
has also used arrows to mark the change in weather when the sun is disappearing 
and the clouds are coming into the picture). Apart from that, images are used to 
show what the pupils found during the excursion, while in words commenting on 
things such as the fact that a snail stuck to everything (“I have found a snail which 
stuck to everything” No. “Jeg har funet en snegle som sugde sig fast på alt”). Such 
information is hard to give through the image. 

In the experimental report, the chemical formula that is written on top of the page is 
only implicitly connected to other parts of the content. Anyone who knows chemistry 
knows that “Mg” corresponds to the word magnesium which is given further down 
in the text, under the heading “The task” (Sw. Uppgiften), and that “HCL” (the gas 
hydrogen chloride) in this case must correspond to “an acid” (Sw. en syra) (HCI) in 
the same section. Furthermore, subject knowledge in the area of chemistry is also 
needed to be able to grasp that the formula only concerns one of the acids that are 
mentioned under “We need” (Sw. Vi behöver), here the solution hydrochloric acid 
(Sw. saltsyra) which corresponds to the chemical bond hydrogen chloride dissolved 
in water. The fact that the image is probably linked to the formula is not made clear 
(the formula should be the theoretical connection to the actual experiment). Hence, 
there is some degree of correspondence between the different resources, though on 
the whole this correspondence is only implicit. 

Below the image of the experimental equipment, the text says “This is what 
we did:” (Sw. Så hdr gjorde vi) with an arrow pointing towards the drawing. The 
information given through writing here implies a description of a process. However, 
the drawing is implicit as to how the experiment was actually performed, and only a 
person who was present during the experimental work (or who knows how this kind 
of experiment is normally performed) will be able to understand what was done. On 
the other hand, the drawing works fairly well to illustrate what equipment was used, 
and how the different parts of the equipment were connected. 

Thus, the drawing works well in relation to its potential for meaning-making, or 
affordance, in this case to illustrate spatial relations and to depict concrete objects. 
On the other hand, the caption “This is what we did” implies a description of a 
procedure, something which would have been easier to express through writing. 
Further down in the written text, under the heading “We need” (Sw. Vi behöver), 
some of the equipment depicted in the drawing is mentioned: “test tube”, “burner”, 
etc. Figures 5.1 and 5.2 in the drawing are not explained, and they do not appear in 
writing. Hence, we note that there is some coherence in the text, but the connection 
between the content given through the different resources is implicit. 

Since we do not know the context in which the experimental report was produced, 
we do not know for sure who the intended reader of the text is. Presumably, it is written 
for the student himself, and if so, most likely to be used in preparations for a final test. 
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Or perhaps it is supposed to be handed in to the teacher for assessment; or possibly 
a combination of the two. 

Regarding the use of chemical symbols, the text is problematic if it is intended to 
be used as a basis for understanding the content area, since the chemical formula is 
not explicitly explained. For the same reasons, its usefulness can be doubted, if the 
text is intended to be used for assessing the student’s content knowledge. 

Regarding the image in relation to other resources, the text can perhaps be used 
to remember how the experiment was performed (however, here it is unclear what 
the numbers 1 and 2 stand for). If used as an assessment of how the experiment was 
performed, or as an instruction to someone else who is about to perform an equivalent 
experiment, it is less functional. 

Furthermore, the written text presented in bullets is not unproblematic either, as 
regards content or genre expectations. What is given under the heading “Results” 
(Sw. Resultat) is expected to be connected to the actual experiment, for example, 
“there was a strong reaction in which oxyhydrogen was formed”. Instead, the student 
wrote “I thought that the experiment was fun, and it went well”. However, under the 
heading “Conclusion” (Sw. Slutsats), the student states that “the reaction between 
magnesium and the oxygen in the air is very explosive”, which is more in line with 
what we can expect from the genre. 


5.4.3 Figurative Language and Values 


In this particular text, an analysis of figurative language or values is irrelevant. 
Regarding figurative language, we may note that such expressions are not used (in 
words or images), and values are hardly expected in an experimental report. Yet, 
values are relevant in science, too, for example in relation to the use of natural 
resources or if it is reasonable to kill animals in order to use them for dissection in 
the classroom. 

In analyses that have previously been performed on student texts from chemistry 
classrooms (Danielsson 2011) and in lower school grades (e.g. Kress 2003), the use 
of figurative language has been noted. In those cases, the figurative language, such 
as “Noble gases are contented” or “the structure of the cells in the onion is similar 
to that of a brick wall” were inspired by the teachers’ oral expositions. If figurative 
language is used in class, it is important to evaluate whether the metaphors used are 
in line with the metaphors used in the disciplinary discourse, as well as whether an 
analogy seems to be both relevant and transparent. 


5.4.4 The Subject Content 


From the perspective of the school subject, the subject content given in texts is 
central, and it is therefore natural to focus particularly on content when working 
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with text creation in different subjects. In the experimental report, we have noted 
that many resources have been used in adequate ways, for example regarding genre 
expectations. Yet we still cannot be sure whether the student possesses the content 
knowledge that the teaching and learning activities staged by the teacher probably 
aimed at. The fact that the chemical symbols are used in line with subject conventions 
does not automatically imply that the student also knows the content behind the 
symbols. 

Under the heading “The task” (Sw. Uppgiften), the student has written “What 
happens when an acid reacts with magnesium?” (Sw. “Vad händer när en syra 
reagerar med magnesium”), which is most likely the same expression as was used for 
this particular task. If connected to the use of chemical symbols, it could very well 
be that the student knows that “HCL” (correctly written: HCI) is an acid (probably 
the solution, if taken into account what is written under “We need”, since hydrogen 
chloride, Sw. véiteklorid, is a gas). But from the text it is not clear what acid “HCL” 
corresponds to (three different acids are mentioned in writing). Also, someone who 
knows chemistry can conclude that the chemical equation on top of the page is 
not balanced. The formula does not reveal how the sulfur ion was formed, or what 
happened to the chloride. 


5.4.5 Developing Multimodal Text Competence 


The analysis of the student text in Fig. 5.1 reveals that this student possesses a good 
knowledge of the structure of the experimental (school) report at a general level. 
When working with the development of this student’s multimodal competence, this 
knowledge would be a natural point of departure. 

A subsequent step could be to discuss in more detail what semiotic mode or specific 
text resource is best fit to express different aspects of the task at hand. One example 
could be to ask the student to reflect upon the use of a single image to describe the 
process “this is what we did”. In this case, some information would clearly be needed 
to be given through words. Also, the text could be used as a basis for pondering on 
ways of making the text even clearer regarding the ways in which different text 
resources interact. One such example could be to complete the chemical equation 
with a description of what it shows, given through words. Here it is important to 
discuss different choices as to semiotic mode, or as to details of the text resource 
chosen in relation to the function of the text, and in relation to the intended reader. 
If discussing multimodal aspects of texts in such ways, the student will get tools 
to enhance the text in ways that will better show the teacher his understanding of 
content, as well as for himself when producing texts that can be used for developing 
his subject knowledge. 

When using the student’s text to discuss the content, it will most likely be evident 
to both the teacher and the student in what ways the student might need to develop 
his content knowledge in relation to the experiment, and to what extent he “just” 
needs to develop his multimodal text competence. Hence, these two aspects are 
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interdependent. In classrooms practices where teachers support meaning-making 
in relation to multimodal texts, students will be supported in their development of 
content knowledge at the same time. 


5.5 Summary 


The multimodal text perspective has a direct relevance for students” own text produc- 
tion. Just as we need a deeper understanding of the ways in which students interpret 
multimodal texts, there is a need for a deepened understanding of multimodal writing 
practices and how students can be supported in their own writing development in 
relation to multimodality. 

In schools numerous writing activities take place, such as memos, lab reports, logs, 
PowerPoint presentations, web pages, posters, and essays. These writing activities 
are related to subject content and the different genres or text types in the area. The 
lab report is usually relatively standardized, while other types of presentations of 
subject content, or the essay, can be more varied and contain personal reflections. 

In the multimodal text the temporality of the verbal text is mixed with the spatiality 
of the image. Therefore, different kinds of information are given through different 
resources, and to compose a multimodal text thus implies something other than 
composing a verbal text. 

By using our text model for analyzing and discussing an experimental report 
created by a student, we have seen that the student masters the genre at a general 
level. From a superficial point of view, the text looks like the way we are used to see 
such reports. However, by looking closely at the ways in which the student has used 
different semiotic modes and resources and how they interplay, potential obstacles 
for the student were revealed. We suggest that teachers proceed from what the student 
masters when discussing student texts. From there, they can discuss how the different 
semiotic resources are used (images, chemical language, verbal language, etc.) and 
why. With such a discussion, focusing on text and content, possible challenges for 
the student regarding the content can be revealed at the same time as the student’s 
multimodal competence can be enhanced. 

The ways of producing and handling texts in the digital world brings other aspects 
to the fore, as when other texts, images, or films are merged in new texts. As a teacher, 
you need to consider how to deal with references to other sources. Such questions 
will be even more important when new ways of using computers in schools become 
increasingly common, for instance in so-called “one-to-many” projects, where the 
students themselves can choose what digital resources to use to meet the standards 
in school. 
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Part II 
Sample Texts Analyses 


Chapter 6 R) 
Close Reading of Multimodal get 
Texts—Sample Analyses 


In the second part of the book we use our model as analytical tool for texts taken 
from various school subjects. Here we aim at revealing how a systematic close 
reading can be used for “wrapping up” a text, in order to make visible how different 
semiotic resources are used in ways that can both be facilitating and challenging in 
the meaning-making process around the content. However, we also want to show 
how teachers can work with a text to make it more accessible and easier to grasp 
for the reader (in this case, the student). In the analyses we focus more closely on 
different aspects of the model depending on what we find especially interesting or 
challenging in the various texts. Also, what the teacher might want to highlight in 
metatextual discussions must vary, and the same text might be discussed in many 
different ways, depending for instance on how experienced the students are when it 
comes to metatextual discussions and multimodal aspects of meaning-making. In the 
following we give some examples as to what can be highlighted in text discussions. 
Such comments are either given under a separate heading or in parallel with the 
analysis. However, these comments should be seen as suggestions and they are not 
extensive or given in relation to each and every text. In classroom practices, the 
teacher will choose what features to focus on in the texts used. Our suggestions as 
to what to discuss can then function as an inspiration. 

This type of close reading will partly be very detailed, and it might acutally be 
perceived as “over-critical”. However, it is not our intention to be critical in negative 
ways. Instead, we want to point out aspects of texts that are relevant for all kinds of 
multimodal texts. To write a text and to make decisions about the layout and choice 
of text resources is challenging, and a number of choices need to be made. Each 
choice will result in both gains and losses, something which is inevitable. But when 
teachers have a more robust approach to multimodal texts, they can better support 
students in working themselves through texts. 
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We suggest that you start reading the sections that deal with subjects that catch 
your interest. You can consider the sample analyses as a supply of resources for 
inspiration. Our examples are never fully exhaustive, and with the help of the model 
you might be able to find further interesting aspects that we might have neglected. 

As already mentioned in the Introduction section, in our selection of texts for 
analyses, we have striven to include as many subject areas as possible, and we have 
used texts from different school levels, even though the emphasis is on texts used in 
upper elementary and lower secondary school. During these school years, texts are 
relatively complex. For lower school grades, texts in science are relatively similar to 
those analyzed here, even though they are less complex regarding verbal aspects (less 
complex clauses and terminology, etc.). Focusing on what information is especially 
emphasized through writing or image, as well as the use of figurative language, or 
values in texts can just as well be carried out in lower elementary school grades. 
The textbooks in upper secondary school, on the other hand, are of course more 
complex and they have a deeper content, but at the same time they do not differ from 
texts in lower secondary school in major ways. Also, our examples are mainly based 
on printed books. One reason for that choice is that printed textbooks (or digital 
texts based on printed versions) still dominate the market in many countries. Here, 
however, we can expect a shift in the near future. As regards examples from different 
countries and cultures, we have tried to find a variety. Since this book is written in 
English, quite a few of the examples have been taken from English-speaking countries 
(in this case the UK and Singapore), but we also use examples from Chile, Spain, 
and Sweden. We ourselves are Swedish, and thus we have access to a variety of 
teaching resources as well as an insight into the educational system. When examples 
with other languages than English are used, we provide the reader with translations 
of central parts of the texts. 

The aim of the close reading of these texts is to provide the reader with detailed 
examples from a variety of school subjects. Yet this does not mean that the close 
reading is fully exhaustive. Our model is dynamic in the sense that different learning 
resources can be analyzed by means of the concepts used in the model. As said, this 
does not mean that there is one, and only one, way of analyzing or working with 
these texts or working with students’ interpretations of them. Our close reading of 
these texts should be seen as a kind of modeled reading, where you can choose to put 
more or less emphasis on the different aspects, depending on the intention behind 
choice of text, or your planned meta-textual classroom discussions. 
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Chapter 7 A) 
Natural Sciences oheck for 


7.1 Physics, Elementary School 


We have chosen a section dealing with the “the celestial bodies in the solar system, 
and their movements in relation to each other” as an example of a text in science for 
elementary school (students aged around 10 to 12 years). This particular text deals 
with how to explain day, night, months, years, and seasons, which is a central content 
area for Swedish school children of this age. Here we will make a close reading of 
a couple of pages in the textbook PULSFysik och Kemi (Sjöberg and Öberg 2011) 
(Eng. “Physics and Chemistry”), and a chapter called “Astronomi—ett himla stort 
ämne” (approximate translation into English to preserve a wordplay: “Astronomy—a 
subject full of space”). 

To start with, we can conclude that to be able to grasp this content, the student 
must have an abstract understanding of the celestial bodies that we can experience in 
a concrete way in our everyday lives (the earth, the moon, and the sun). Therefore, in 
the close reading of this text, we will mainly concentrate on the general structure and 
setting and the interaction between parts of the text. We will, however, also comment 
briefly on figurative language presented in writing, since metaphors are central for 
science. 


7.1.1 General Structure and Setting 


Together, the images in Figs. 7.1 and 7.2 constitute an introductory book spread of 
the chapter. As regards the setting of the spread, we may note that the pages consist of 
a variety of text resources (various headings, an introductory paragraph, running text, 
etc.). The illustrations, comprising a photograph and some drawings, are prominent, 
and they take up a large part of the book pages. The drawings show, on the one hand, 
children (they reappear in the whole textbook, as interested experiencers, thinkers, 
etc.), on the other hand schematic images of the movements of celestial bodies. 
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ASTRONOMY - a subject 
full of space 


ASTRONOMI 


Since ancient times, we 
have used the stars, the 
sun and the moon to keep 
track of the time. The sun 
rises in the morning, 
moves across the sky 
during the day, and sets at 
night. In this chapter, we 
will take a closer look at 


—ett himla stort ämne 


Sedan urminnes tider har vi använt stjärnorna, solen 
och månen för att hålla reda på tiden. Solen går upp 
på morgonen, förflyttar sig över himlen under dagen 

och går ner på kvällen. | det här kapitlet ska vi se närmare 
på hur solen, jorden och månen rör sig | förhållande till 
varandra, varför vi får dag och natt och hur det blir olika 
årstider. Vi ska också bekanta oss med andra delar 


av solsystemet. 

how the sun, the earth, and 
Solen, månen, stjärnorna och planeterna som du kan se på the moon move in relation 
himlen är desamma som människor har sett i alla tider. Många 
stjärnbilder har namn som är flera tusen år gamla. Människor P E E ENE E tt to each other, why we get 
har tänkt sig olika sätt att förklara hur himlakropparna kan röra solen Nir månen kommer 3 
sig som de gör. Länge tänkte man sig att jorden var världens mellan solen och jorden bil day and night, and how 
mittpunkt, och att allt annat rörde sig kring jorden salts diacaa bara ee ER different seasons arise. We 


flack pa jorden. 


will also become familiar 
with other parts of the 
solar system. 
(approximate translation 
of the bolded introductory 
paragraph) 


Fig. 7.1 Astronomy, elementary school. PULS. Fysik och kemi 4-6 grundbok (Sjöberg and Oberg 
2011, p. 90, illustrations by Jenny Karlsson; re-printed with permission from Natur & Kultur) 


Even from a quick description like this, it is evident that these are quite typical 
book pages considering the subject and the intended readers. We can also note that 
illustrations appear to be important, both for illustrating the subject-specific content, 
such as the drawings of day and night on the earth, and for alleviating the visual 
appearance (the children who are given various roles, at the same time as the readers 
can identify themselves as becoming scientists through them). 

The chapter is structured thematically, in ways that give the reader an overview of 
the content, stated in the introductory paragraph in bold. Here, the point of departure 
is the reader’s presumed everyday experiences of sunrise and sunset. Also, it gives a 
historical perspective, informing the reader that humans through history have used 
the celestial bodies to keep track of time. Such a historical perspective reappears 
in parallel to the scientific content all through the chapter. In Swedish curriculum 
documents, it is commonly stated that all subjects should give a historical perspective 
to the content area in question. The scientific content in this book spread describes 
how the earth is rotating around its axis, and how this rotation regulates day and 
night. 
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Jorden snurrar runt 


Det var Nicolaus Copernicus (1473-1543) från Polen som först 
föreslog att jorden rörde sig runt solen i stället för tvärtom. Det 
var ett djärvt förslag eftersom det inte stämde med kyrkans läror. 

Solen ser ut att röra sig över himlen. Men i själva verket är 
det inte solen utan jorden som rör sig. Den snurrar runt, och 
eftersom vi står på den snurrande jordens yta, ser solen ut att 
röra sig. Och det gäller inte bara solen. Under natten tycks 
månen och hela stjärnhimlen röra sig på samma sätt. 

Jorden roterar ett varv runt sin egen axel på ett dygn. 
Vi kan tänka oss jordens axel som en lång pinne från Nord- 
polen till Sydpolen. 

Bilderna visar hur jordens vridning gör att vi får natt och dag. 


Nu är klockan 9 på 
morgonen, då är klockan 
3 på natten i New York. 


Natt — — Morgon 
i Sverige i Sverige 
Sommartid 
Sommartid innebär att 
vi ställer fram klockan en 
Var vi än är på jordytan rör vi oss åt öster. När platsen där timme under sommar- 
vi är kommer ut ur skuggan ser vi hur solen går upp. Det halvåret. På så sätt kan 
blir ljust. Men så småningom kommer vi in i skuggan igen. vi få njuta av en extra 
Då går solen ner och det blir natt. timme dagsljus på 


kvällen. 


Olika slags tid 


Ju längre österut du befinner dig, desto senare på dagen har 

det hunnit bli. Du vet säkert att klockan kan vara olika mycket 
i olika länder. Men även inom Sverige märks det skillnad. Solen 
går upp lite tidigare i Stockholm än i Göteborg. 

I mitten av 1800-talet började man använda ångbåtar och 
tåg och behövde därför trycka tidtabeller. Då var det opraktiskt 
att ha olika tid i olika delar av landet. År 1879 införde man 
därför normaltid, som är lika i hela Sverige. 


91 


Fig. 7.2 The earth is spinning around. PULS. Fysik och kemi 4-6 grundbok (Sjöberg and Öberg 
2011. 91, illustrations by Jenny Karlsson; re-printed with permission from Natur & Kultur) 


7.1.2 Interaction Between Text Resources 


Regarding the interaction between the different parts of the text, a number of aspects 
are worth focusing on, not least in relation to spatial proximity and coherence between 
different types of written texts (text boxes, headings, captions, etc.) and other textual 
representations. The illustrations on the first page are only loosely connected to the 
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introductory paragraph and the running text. The only clear connection between the 
resources on that page is the one between information in the introductory paragraph 
and the drawing of a squatting girl looking at a sundial. For a reader with no previous 
experience of sundials, this connection might not be clear. 

On the introductory page, there is also a caption, placed between the two illus- 
trations: “The earth and the moon move around the sun. When the moon comes in 
between the sun and the earth, there is a solar eclipse. Here we can see the shadow of 
the moon as a dark patch on the earth.” There is no indication as to what illustration 
the caption refers to. For several reasons, it is possible that the “novice” will interpret 
the caption as part of the illustration of the sundial and the girl. First, that illustration 
is prominent, as regards placement, colors, and composition, while the section from 
the satellite image is diffuse and may be difficult to interpret. In the first sentence of 
the caption, three elements are mentioned. For the novice who is not yet familiar with 
the content, these three elements can be connected to three prominent parts of the 
illustration with the child and the sundial, namely, the sun, the child, and the sundial. 
To be able to read three elements into the section of the satellite image, the reader 
must imagine both the moon and the sun, and also their placement in relation to each 
other. The part of the text in the caption that says “Here we can see the shadow of the 
moon as a dark patch on the earth” (p. 90) clarifies that the text relates to the satellite 
image, and it is also clarified that the image depicts the earth. It is worth noting that 
the caption contains information about how solar eclipses arise, though this is not 
mentioned anywhere else in the chapter. 

As said, the diffuse section of the satellite image can be challenging to interpret. 
However, using a photograph, which in this case is intended to show a solar eclipse, 
is an opportunity to depict a phenomenon which could be hard to grasp otherwise 
(the fact that the moon casts a shadow on the earth when placed between the sun and 
the earth). If the caption had been more explicit about the fact that the image was in 
fact a satellite image of the earth seen from far away, this could have supported the 
student trying to interpret the image. 

The next page deals with the earth’s rotation around its axis. Here the illustration 
has a concretizing function, where day and night are represented by two globes. 
Combined with the two stylized globes is an analogy that the earth rotates around an 
imaginary axis. In this case, the rotation around the axis is concretized through a stick 
supplemented with a curved arrow pointing in the direction of the rotation. Through 
this illustration, the student is supposed to imagine that the two stylized globes 
represent the earth at two different times, but also that the stick does not represent 
a concrete physical phenomenon. Instead, the reader is supposed to understand that 
the stick has been added as a way of visualizing the tilt of the earth in relation to the 
sun, as well as the way in which the earth rotates. Thus, the reader is supposed to 
not to take the stick into account as a real object. Another complication is the fact 
that the tilt of the earth is not real—there is no up and down in the infinite space to 
relate to. Instead this tilt concerns the relation between the earth and the sun, which, 
by the way, is not present in the illustration. 

The congruence between the information given in writing and that given in illus- 
trations is more apparent than on the previous page. First, in the running text there is 
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an explicit connection to the illustration through the words “The images show how 
the rotation of the earth gives us night and day” (approximate translation, p. 91). 
Second, in writing, too, the analogy between the axis of the earth and that of the stick 
going from the North Pole and the South Pole is made explicit. 

Apart from the assumption that the tilt of the earth should be seen in relation to the 
sun, there are other potential challenges both in illustrations and writing, and in the 
relationship between them. Here we will comment on a couple of such challenges. In 
the illustration there is a slightly curved arrow, a conventionalized symbol to depict a 
direction or movement.! The arrow points towards the right and is therefore intended 
to show the direction of the rotation of the earth. In this case, the reader is supposed 
to make the connection between the arrow and a comment in the caption that we 
always are moving towards the east, as well as information in the running text about 
“the twist of the earth” (Sw. jordens vridning). The caption also says that we are 
either in or outside of the “shadow” (Sw. skuggan). How this shadow emerges is not 
made explicit, however, neither in writing nor in the illustration. Instead, the reader 
must imagine the sun and its position in relation to the earth. Furthermore, the reader 
is supposed to understand that the illustration is not supposed to depict two different 
globes, but instead that it is supposed to depict the earth at different points of time. 

On the same page, there is also a toned text box. Text boxes are often used as a 
kind of “short facts” in pedagogical texts where the reader can expect to find key 
content. Here, however, the text box gives some “extra information”. In this case the 
information deals with summer- and winter-time, which is used in large parts of the 
world, although this fact is not taken up in illustrations or in the running text. 


7.1.3 Figurative Language 


We have mentioned the use of an analogy between the axis (in itself a figurative 
description) of the earth and a stick through the globe. Here the authors have chosen 
to integrate the analogy in both the running text and the illustration. The choice to 
actually draw the analogical axis could be seen as a way of concretizing natural 
phenomena in science, to facilitate learning. In this case, in writing it is clearly 
expressed, through an explicite simile, that this is an analogy “we can think of [...] 
as a stick”, while in the illustration the depiction of the stick is integrated as a natural 
part of the earth. 

It may be doubted whether the fact that the axis of the earth specified as a stick 
actually facilitates meaning-making related to the analogy. Instead, it is possible 
that the illustration makes the analogy more challenging, since it is unnecessarily 
concrete. Yet the fact that the analogy is explicitly explained in writing may be 
assumed to facilitate meaning-making for the student. The risk of coming to the 
wrong conclusion, that there is actually some kind of stick going through the globe, 
will then be diminished. 


' Among other things, an arrow can also depict a process, such as the transformation from one thing 
to another, or a temporal relationship between entities. 
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The conventionalized use of “axis’”—a metaphor which is integrated in scientific 
discourse—is less clear in the text. There is in fact no axis, but instead the term is used 
as if there was an axis going from one pole to another. Yet another complication is that 
in Swedish the word axis is homonymous with the word for shoulder (Sw. axel). Ina 
Swedish context, it is likely that children of this age are only familiar with the word 
axel as part of the body. This is an example of how subject-specific terminology with 
another use in everyday language can be a challenge for meaning-making around 
subject content. It is especially important to be aware of such challenges regarding 
students who learn in a second or foreign language. 


7.1.4 Classroom Focus 


From an educational point of view, a number of aspects in this text are worth devoting 
attention to in the classroom. Already the general structure of the layout and the 
graphical choices open up for things to discuss in the classroom, in ways that can 
function as a model for how to approach new texts. A starting point can be a mutual 
examination of illustrations and text elements on the pages. In relation to such a 
general overview, the teacher can point to the usefulness of benefiting from one’s 
previous ideas and knowledge about text resources such as introductory paragraphs, 
headings, text boxes, illustrations, or captions. It may be worth noting that such 
textual resources can be used in similar ways across texts, but also that they can have 
different functions in different texts. 

With regard to the various illustrations, a starting point can be to try to make 
out what they might depict, and how the illustrations can connect to the content in 
question, in this case “astronomy”. Our close reading above pointed out a number 
of potential challenges regarding the interaction between different text resources. 
One way of dealing with that could be to read the caption together to find out which 
illustration the caption relates to, and how to arrive at that conclusion. 

For all of the illustrations that are connected to the subject content, it is important 
to examine how the running text and captions are connected to the illustrations. One 
aspect to highlight can be what is expressed through the different resources and 
if they give the same (overlapping), or competing, or perhaps even contradictory, 
information. In all texts where figurative language is used (in this case the axis of 
the earth which is compared to a stick), it is worthwhile reasoning about how far the 
analogy reaches: which parts of the analogy are relevant and which are not? 


7.2 Biology, Secondary School 


Here we will examine another Swedish textbook, Spektrum Biologi (Fabricius et al. 
2011) which is a fairly extensive book, dealing with areas like “Life on earth”, “Bac- 
teria and viruses”, “Ethology”, “Our environment’, and “Drugs”. We have examined 
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a couple of pages from the section “The human body”. In a section on circulation in 
the human body we find the heading “The organs cooperate” (Sw. Organen samar- 
betar). At the left side of the book spread, the different organ systems are described. 
Here we take a closer look at the right side of the book spread, which is shown in 
Fig. 7.3. 


MANNISKOKROPPEN 


Sa halls var kropp igang 

Kroppens organsystem samarbetar hela tiden for att 
hålla igång var kropp. För att se hur det kan fungera 
gor vi en snabbresa genom fyra av organsystemen — in 
och ut ur kroppen. Du kan följa resan med hjälp av 
den förenklade bilden till höger. 

För att vår kropp med alla sina tiotusentals miljar- 
der celler ska fungera behöver den syre och närings- 
ämnen. Andningsorganen kan ta syre ur luften och föra 
över det till blodet. Matspjälkningsorganen sönderdelar 
maten så att näringsämnena kan tas upp av blodet. 

När de här organsystemen gjort sitt jobb är det 
dags att transportera näringsämnena och syret till alla 
celler i kroppen. Det tar cirkulationsorganen — hjärtat 
och blodkärlen — hand om. Med deras hjälp fir alla 
celler det som de behöver. 

När cellerna sedan använder de olika näringsäm- 
nena och syret bildas ämnen som inte behövs. De 
måste kroppen kunna göra sig av med. Det sköts av 
utsöndringsorganen — njurarna och levern. De ser till 
att avfallet utsöndras med urin och avföring. 


Det fantastiska maskineriet människokroppen Organsystemen samarbetar för att 


bygger alltså på en ständig samverkan mellan organ- kroppens alla celler ska få syre och näring 
samt bli av med det avfall som bildas. 


systemen. 


Cellerna "pratar" med varandra 
med hjälp av signalämnen som 


skickas med blodet = \ 


Meddelanden med blodet 
Kroppens celler och organ kan samarbeta genom att skicka 
meddelanden med blodet. Sådana meddelanden skickas 
med hjälp av signalämnen. Via små blodkärl kan alla celler 
och organ nås av olika signalämnen, till exempel bormoner. 
Hela tiden skickas en mängd olika signalämnen till och 
ipaa från cellerna. De kan till exempel få cellen att tillverka mer 
meddelandet in i cellkärnan av ett visst ämne eller få den att dela sig snabbare. 


$. 


Fig. 7.3 The human body as a system. Spektrum Biologi (Fabricius et al. 2011, p. 247, with 
permission from Liber) 
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7.2.1 General Structure and Setting 


Just like the bottom part of the page, the upper part, which dominates the page, 
consists of writing combined with an image. Both images on this page are prominent, 
the upper one perhaps because of its shape and the red color. To the left of this 
illustration, under the heading “This is how our body works” (Sw. Så halls var kropp 
igång) a quite long written text dealing with the ways in which the organic systems of 
the body “cooperate”. This text is connected to a gray, egg-shaped illustration to the 
right. The image depicts four organic systems in quite an abstract fashion: digestion, 
perspiration, circulation, and secretion. The blood system is mainly depicted in blue, 
as opposed to the red color of the blood in the image at the bottom of the page. The 
gut and the intestines have, for some reason, the same color as the body cells. 

At the lower part of the page, to the right, there is a short written text about “signal 
substances” under the heading “Messages with the blood” (Sw. Meddelanden med 
blodet). The wording of the heading and what it actually relates to is unclear. The 
illustration to the left of the text depicts a section of a blood vessel which looks like 
a red flood, with blue cells constituting some kind of end points. 

What are the most prominent parts of the text on this page? Different emphasis 
is shown in different resources. In the upper part of the written text to the left, all 
of the different organic systems are italicized (perspiration, circulation, etc.). In 
the illustration to the right, on the other hand, the heart and blood circulation are 
centrally placed. Their prominence is further enhanced through the bright blue color. 
The other organs have been placed on the periphery of the “egg”, in a somewhat paler 
pinkish-brown tone, green, and lilac, respectively. The fact that the blood system is 
prominent on this page is further enhanced by the vivid red color of the blood vessel 
at the lower part of the page. 

We may presume that the author and the illustrator of this text want to enhance 
aspects that they perceive as especially prominent or important, within the different 
“systems”. Naturally, not everything can be told or described, but the selection of 
information—in the form of facts in writing and illustrations—is not particularly 
easy to interpret in this text. Therefore, let us now go on to examine the interaction 
between different textual resources. 


7.2.2 Interaction Between Text Resources 


As mentioned previously, illustrations can be both facilitating or challenging for 
making sense of a text. In one sense, the “egg” in the upper right part of the page 
is concrete, since it depicts the interaction between systems in our body. But at the 
same time, the illustration is highly abstract, since it does not show what the organs 
look like, or their placement in relation to each other. Instead, the focus is on how 
the “systems cooperate”. 
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The various systems have been placed side by side, with relative proportions which 
bear no resemblance to their real size. The small and the large intestines are placed at 
one side of the body, while the heart and the blood circulation system are shown as a 
more or less closed system in the middle of the illustration. Arrows placed between 
the blood circulation system and, for example, a kidney or intestine, reveal that there 
is some kind of connection or interchange between the parts. The body cells—where 
the size of each cell corresponds to approximately a fifth of a kidney—have been 
placed to the right in the illustration. Through their placement and heading style, 
they appear to be one organic system among the others. 

From the right part of the illustration, it seems as if nutrition and secretion are 
connected, on the one hand, to the body cells, on the other hand, to the heart and blood 
circulation. At the same time, we find a digestive organ to the left, where something 
which gets into the intestines from the outside disappears as excrements down to the 
left. However, a kidney which is connected to the blood circulation through an arrow, 
and an arrow pointing towards the word “urine” (Sw. urin) can be found down to the 
right, though they do not seem to be connected to any kind of food or liquid intake. 
Eventually, the reader might realize that what is called the “secretion organ” (Sw. 
utsOndringsorgan) has something to do with the purification of the blood. 

In the illustration, the word “blood circulation” (Sw. blodomlopp) is given, though 
in the text the word “blood vessel” (Sw. blodkärl) is used. Where the text mentions 
the circulation organ “heart—blood vessels”, the illustration depicts the circulation 
organ “heart—blood circulation”. Thus, here the text is in conflict with the principle 
of congruence. 

Not only form, but color, too, carries information in the illustration. In this case, it 
can result in further challenges in the meaning-making process. The digestion organs 
to the left are depicted in a pinkish-brown color, just like the cells to the right. Here 
there is a congruence in the color coding, but does this mean that there is a connection 
between the digestion organs and the body cells? In line with the conventions of the 
particular subject area, it can be assumed that the red color depicts blood which is 
rich in oxygen, while blue depicts blood with a low level of oxygen. 

The lungs have a pale lilac tone, while the kidney (why just one?) is light green. 
However, what does the gray shade in the background mean? Is it presumed to be a 
neutral background meant to separate the illustration from the white book page, or is 
it a depiction of “the body”? Thus, we can state here that even though color to some 
extent appears to carry meaning, this is not always the case. 

On the whole, we may note that the relation between writing and image is complex 
in this text. Since the illustration in itself is quite abstract, the reader needs to have 
quite an advanced content knowledge to be able to interpret the illustration in rela- 
tion to the fact-oriented information given in writing. The connection between the 
information in image and writing might be simple to make for anyone who is already 
familiar with the function of the organs of the body. However, for someone who comes 
across a text like this without previous knowledge, the information can appear to be 
incomplete, and to some extent even contradictory. 
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7.2.3 Figurative Language 


This text contains an abundance of figurative language, not least metaphors, perhaps 
due to a presumption that the reader will be supported through connections to 
everyday experiences. However, such usage can also be potentially challenging. The 
text says that “the organs cooperate”, and “[w]hen the digestive organ have done 
their job, it is time to transport the nutrition and oxygen to all body cells”, which is 
“taken care of” by the circulation system”. But what, for instance, does it mean that 
the digestive organs have “done their job”? And are the circulation organs waiting 
for a delivery from the digestive organ? 

Further, the text says that the body is “a fantastic machinery” (Sw. ett fantastiskt 
maskineri). But a biological body is an organism governed by other principles than 
mechanical clocks or digital systems in a computer. In line with this, researchers in 
the area of brain research have left the analogy of the brain as a computer. Instead 
the brain is discussed in terms of a live organism which is dependent on a biological 
body which actively chooses and sorts information (Edelman 2006). Some adequate 
questions to pose in relation to this text are, for example: What are the gains of using 
the machine metaphor for the body, and what are the limitations? What possible 
misunderstandings can the machine metaphor give rise to? 

At the bottom of the page, to the left, there is an illustration depicting how cells 
“talk” (Sw. pratar) to each other, a metaphor in line with the 1940s communicative 
model by Shannon and Weavers (1998). Here it is said that signal substances are sent 
to the other side, where there is a “receiver [...] who takes the signal substances into 
nucleus of the cell”. Here, the signal substances have become “messages” which are 
actively “taken care of” by a “receiver”, who in turn “brings” the message into the 
nucleus of the cell (cf Pettersson et al. 2020, who showed how students used logistics 
metaphors to talk about the digestive system in a national test). The text humanizes the 
organs, thus depicting the body parts as having intentions. Such anthropomorphisms, 
as they are called, are known to create challenges for students, since the students 
sometimes mistakenly perceive natural phenomena as actually having intentions and 
feelings (for further discussions, see for instance Danielsson et al. 2018; Tibell and 
Rundgren 2010). 


7.2.4 Values 


In this text, no values are expressed explicitly. But are there any implicit values? In 
one sense, you can say that there is an underlying principle for organizing the subject 
content: a functionalistic model with cooperating entities possessing intentions. 

Of course, it might be reasonable to use metaphors to present the function of our 
organs to the students. At the same time, such use can lead to other obstacles since 
the organs and their different parts are depicted as active objects. In one sense, such 
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a depiction weakens the value of the system model presented at the beginning of this 
section of the textbook. 
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Chapter 8 A) 
Social Sciences Crock for 


8.1 Geography 


Geography is a school subject that embraces both natural sciences and social sciences, 
as well as the crossroads between the two. A variety of multimodal representations 
have long been used in geography textbooks, such as different kinds of maps of nation 
states and cities, weather conditions and natural resources, or production and trade. 
Today we can also find digital representations of dynamic changes, for example, the 
development of the earth or the changes of contemporary local landscapes. 

It is interesting to note that since a majority of the population on earth today live 
in urban areas, the city itself has become an important part of what geography is 
about—such as questions concerning the planning of urban development and the 
need to foresee, for example, overflow damage to the city structure. In the following, 
we will give examples from one Spanish and one Singaporean textbook. 


8.2 Geography—A Spanish City: Madrid 


In the following we will present a short textbook description of Madrid as an 
autonomous region, based on a bookspread taken from Tierra 5. Medio Natural, 
Social y Cultural (Botines et al. 2006) (Fig. 8.1). 


8.2.1 General Structure and Setting 


This book spread describes “Madrid, an autonomous region” (Sp. Madrid, una comu- 
nidad autónoma) (1 in Fig. 8.1.). On the left page, the main textis presented in the left 
column (2, 3, 4). The authors describe Spain and its 17 autonomous regions according 
to the legislation from 1978 (2) and that Madrid gained its independence 1983 (3). 
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Fig. 8.1 Schematic image of Madrid, Tierra 5 (Botines et al. 2006, pp. 178-179). Image changed 
for copyright reasons 


The authors then describe different kinds of services available to the citizens, such 
as health care, schooling, and transports (4). In the right column, a green-shaded 
text box presents the legislative statutes for Madrid as an independent region (5). 
Underneath is a photo of houses and streets in central Madrid (7), with a caption 
explaining that this is a big city hosting most of the central institutions and official 
services (of the region as well as of the whole of Spain) (8). 

On the right page, in the left column, we find two different illustrations. On top, 
there is a map of all the autonomous regions in Spain (10), and underneath a diagram 
with illustrated examples of public services financed by taxes (11): education (12), 
health care (13), public transport (14), communal parks (15), and sports arenas (16). 
A caption underneath the diagram says that a large part of the taxes go to public 
services (17). A yellow-shaded textbox to the right consists of a number of questions 
divided in three different areas: “Study the map”, “Comment”, and “Respond” (18- 
20). The first area is connected to the map (18), the second one to the image of 
the network of services (19), and the third one consists of questions, the answers to 
which can be found anywhere in the text (20). 


8.2.2 Interaction Between Text Resources 


In this textbook, we note a fairly high degree of proximity between different text 
resources. In the example used here, on the left page there is a written text about 
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autonomy, illustrated by an excerpt from the by-laws of the city (6). The description 
of Madrid is illustrated by a photo of the city (7), and the description of the city given 
in writing at the bottom of the page is connected to the diagram at the bottom of the 
subsequent page (11-17). 

On the right page, the map (10) is linked to different questions under the heading 
“Observe the map” (Spa. Observa el mapa), explicitly requesting the student to use 
the map in different ways (18). These questions emphasize factual knowledge, such 
as the number of autonomous regions and their names. The diagram on the same 
page (11-17) is linked to questions about the commitments of the city, focusing on 
aspects such as the responsibility of the city (19). The questions on the bottom of the 
page (20) concern facts given somewhere in the book spread. 


8.2.3 Classroom Focus: To Create a Narrative 


In this example, it seems necessary for the teacher to create a narrative in order 
to explain the significance of the different factual pieces of information. Why, for 
example, was 1983 an important year for the city of Madrid, and what were the 
consequences for the city, the region, and the country? In doing this, the teacher can 
link the narrative to the illustrations on the right-hand page, for example to clarify 
public services (with the aid of the diagram at the bottom of the page), and also to 
compare Madrid as a region with other regions in the country (the upper map). 

In other words, the multimodal resources in this book can be linked together by 
the teacher to present an overarching story about the historical and social significance 
of the information. 

In the next example, which is from Singapore, we may note a narrative structure 
in the textbook, which can result in a more precise engagement of (possible future) 
actions by the student. Here the information is framed in environmental terms, not 
only in terms of political or administrative functions, and the role of the teacher 
might in this case be to support discussions among students, and to discuss different 
solutions to complex problems. 


8.3 Geography—Urban Living: Singapore 


The following example is from the textbook All About Geography, Urban Living 
(Goh et al. 2015), where we will focus on the example “Floods. How can cities 
prepare for flood?”. 
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8.3.1 General Structure and Setting 


We here begin our analysis with a book spread about “River floods” and “Flash 
floods” (Fig. 8.2). At the top of the left page we find a drawing—a cross-section of a 
river channel, and at the bottom two different aerial photos of the Mississippi River 
near Memphis in normal conditions (to the left) and during the flood in 2011. 

The top of the right page shows a photo taken from the air during the flood in 
Brisbane, 2011, and at the bottom a flash flood in Kuala Lumpur, 2012. In the right 
margin there is an explanation of “Geographical Concepts—Scale” such as large- 
scale floods (Brisbane in 2011) and small-scale floods (the flash flood in Orchard 
Road, Singapore, 2010). Here there is also a “Bookmark” with a link to a video film 
about “the experience of people during a flash flood”. 

Most of the space is dominated by visual illustrations, which underline severe 
effects of flooding throughout the world. These pages also underline that a flood can 
happen suddenly, and for different reasons. 

On the following pages there are descriptions of different causes of floods, such as 
rainfall (intensity, duration, frequency), snowmelt, storm surge, and tsunami, height 
of land above or below sea level and proximity to water bodies, or the failure of man- 
made structures (such as dams, levees, and floodwalls, or an under-dimensioned 
channel capacity). Here the reader can also find out how floods affect people living 
in cities, such as the spread of diseases and loss of life, homelessness, disruption of 


River floods 


Flash floods 


Fig. 8.2 River floods and Flash floods. All About Geography, Urban Living (Goh, Yu & M.C. 2015, 
pp. 156-157, Reproduced by permission of Hodder Education, photographs blurred for copyright 
reasons) 
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Fig. 8.3 How should cities prepare for floods? All About Geography, Urban Living (Goh et al. 2015, 
pp. 180-181, Reproduced by permission of Hodder Education, photographs blurred for copyright 
reasons) 


clean water supply, damage to transport infrastructure, or disruption of energy supply. 
This leads to the question how cities then should prepare for floods (Fig. 8.3). 

At the top of the left page in Fig. 8.3 we find a heading and a list of what “you will 
learn”. Underneath we find two photos, one showing a flash flood at Lucky Plaza in 
Singapore in 2010, and the other showing people placing sandbags along a road in 
Bangkok in 2011. On the right page different “mitigation measures” are presented 
such as “Regulation”, here exemplified by “Zoning”. On the following pages we also 
find other examples such as “Elevated properties” and “Investment in infrastructure”, 
“Forecasting and warning system”, and “Evacuation drills”. 


8.3.2 Interaction Between Text Resources 


In our first example from Urban Living (pp. 156-157, Fig. 8.2), the relations between 
visual illustrations and written text are close and congruent. The illustrations are 
the predominant resource, and they can function as a starting point for discussions. 
Central here is the phenomenon “floods”, including different descriptions of “scales”. 
However, through the sketch of the cross-section of a river channel, there is also an 
opportunity to start a discussion about causes (which can be followed up and taken 
deeper later in the book). 
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Our second example (pp. 180-181, Fig. 8.3) is focused more on the socio- 
economic impact and the concern for communities that are affected by natural 
hazards. The central theme here is preparations to handle floods of different kinds, 
with a stronger emphasis on explanatory and descriptive texts. The teacher can of 
course, if necessary, clarify how the information, the descriptions, and explanations 
on the following page can be used to answer the questions on the upper left page 
(p. 180). 


8.4 Social Sciences as a Multimodal Challenge 


We have focused here on two kinds of representations of geography: one more 
traditional textbook focusing on political and administrative legislations, and one 
more modern focusing on overarching themes, which also has references to sources 
outside the book. However, by using these examples, we would not want say that 
the first is “wrong” and the second one is “right”. What we want to underpin here 
is that the different resources present different affordances for the teachers and the 
students, as well as different opportunities to use the information: different resources 
give different opportunities for teaching and learning. In both cases, it is up to the 
teacher to frame the information so that the pupils will have an opportunity for mean- 
ingful learning. Therefore, it is not enough to (only) explain what the words as such 
mean; these words must also be framed in relation to other kinds of multimodal 
representations. 


8.5 History, Secondary School—A Matter of Values 


In our model for reading texts with a focus on multimodal and educational aspects 
(Table 4.1, Sect. 4.5), we differentiate between the overall structure and setting of the 
text, the relation between different parts of the text, as well as the role of figurative 
language and values. We have also stated that you do not have to use all the categories 
every time you analyze a specific text. In our analysis of history textbooks, we would 
like to emphasize the specific role of values, looking at two textbooks about the 
war in Vietnam: the Swedish book Utkik, Historia, Grade 7-9 (in English “Lookout, 
History—History Grade 7-9”, Nilsson et al. 2013) and All About History. The Making 
of the Contemporary World Order, 1870s—1991, Cheong and Huang 2015) from 
Singapore (Fig. 8.4 and 8.5). 
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Vietnamkriget 


Fig. 8.4 The Vietnam War, Utkik Historia (Nilsson et al. 2013, pp. 270-271, with permission from 
Gleerups) 
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Fig. 8.5 How did the Vietnamese succeed in getting the French to relinquish its control over Vietnam 
by 1954?, All About History (Cheong and Huang 2014, pp. 50-51, Reproduced by permission of 
Hodder Education, photograph blurred for copyright reasons) 
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8.5.1 General Structure and Setting 


At a general level, one can immediately notice that the Singaporean book empha- 
sizes “the making of nation states”, whilst the Swedish book more vaguely focuses on 
“the time after the war”. A closer look reveals that the heading in the Swedish book 
announces “Vietnam”, and the subtitles focus on “History and statistics” and “Killed 
Americans in Vietnam” (our emphasis) respectively. In the Singaporean book, the 
heading talks about “Decolonisation of Vietnam”, and the subtitles are phrased as 
follows: “How did the Vietnamese succeed in getting the French to relinquish its 
control over Vietnam by 1954?” (our emphasis): “To what extent was the estab- 
lishment of an independent Vietnam in 1976 more a result of internal rather than 
external factors”? At the end we find the subtitle “Think like a historian: Causation: 
categorising factors” and “Practising history: Developing and concluding a causal 
argument”. 

Both the thematic orientation and the sequencing of information are obviously 
differently designed in these two textbooks. We can also notice that the Swedish book 
introduces the term “domino theory” (i.e. in the political language of those days in 
the West—if one country falls into the hands of the communists, others will follow), 
whilst the Singaporean book instead emphasizes the question of “anti-colonialism”. 

If we now continue with the question of what different semiotic resources offer, 
we will find the following caption in the Swedish textbook: “The Vietnam War’, 
although most of this part talks about the different wars in Southeast Asia. This 
is followed by subheadings and short, descriptive texts. The map on the left page 
(Fig. 8.4, at the place of “the given” according to Kress and van Leuwen 2006) shows 
where in the world we can find Vietnam, whilst the information on the right page 
(“the new” information) shows American soldiers as active agents (those who do 
things), albeit with some negative hints. 

The Singaporean textbook shows on its left page General Giap (at the place of 
“the given”), and on the right page (the “new” information) a subheading about 
“Vietnamese anti-colonial sentiments” (Fig. 8.5). In this book, the world map is not 
that necessary, since those who read the book probably know very well where in the 
world they are situated. Instead, the Vietnamese victory is the given starting point, 
and the Vietnamese people are seen as the active agent. And in addition, under the 
subheading “Background”, we find different “Tasks” for the reader to solve. 


8.5.2 Interaction Between Text Resources 


Regarding the interaction between text resources we may notice interesting differ- 
ences. In the Swedish book, the map is linked to the escalating war in Southeast Asia. 
It starts with the war with “the French” and the battle at Dien Bien Phu, followed 
by the war with “the US”. The photo on the right page (Fig. 8.4) is talked about in 
terms of “severe sufferings”. 
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The text book from Singapore emphasizes to a larger degree the war against the 
French colonial power. On the left (Fig. 8.5) we find, as mentioned, General Giap and 
the Battle of Dien Bien Phu. On the right side, we find different pieces of information: 
tasks, websites (and later also “sources”, for example with quotations from books and 
speeches) and a text that describes the background to the anti-colonial movement. 
The war with the US comes much later in the book. 

In this book, we also find a thematic thread from victories and anti-colonialism to 
the building of new nation states. Even though the Swedish book also talks about “The 
liberation of the colonies” (pp. 268-269, not shown in the figures), it is with a very 
distant voice: “The so-called de-colonization started in Asia at the end of the 1940’s 
and spread to Africa 10 years later”. Furthermore, the perspective of colonization is 
not directly linked to what then follows about the Vietnamese war. 


8.5.3 Figurative Language and Values 


We do not find a lot of explicit figurative language in these texts. However, the 
Swedish book refers to the “domino theory” or “crushing defeat”, and in the Singa- 
porean book expressions like “power vacuum” and “puppet government” are used. 
On the other hand, the values that are brought to the fore are quite obvious, even 
though they are not explicitly stated. For example, the Swedish book emphasizes 
that out of 2.5 million people that were killed during the Vietnam war, 58,000 were 
Americans, and on the following page we find a map showing how many Ameri- 
cans were killed in different provinces. In the book from Singapore it is instead the 
suffering of the Vietnamese people that is emphasized (during both the French and 
the American occupation)—when peasants lost their land and higher education was 
accessible only for students from privileged groups. 

Obviously, the socio-political place in the world frames perspectives and the selec- 
tion of facts—where the western hemisphere gradually seems to lose its control over 
different parts of the world, and where the eastern hemisphere grows in importance 
and self-government, and self-esteem. A way for teachers (in both countries) to high- 
light these aspects could be to compare multimodal representations, descriptions and 
explanations of the “same” historical situation from different points of view, as can be 
found in different learning resources—which also will reveal the underlying (often 
immanent and “natural’’) value statements. 


8.5.4 Representing History 


The discussed differences between the two books can of course also be found in 
other parts in each of them. For example, the captions on the following pages in the 
Swedish book highlight such themes as “Humans on the move”, “The end of the 
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Cold War” and “Towards a global world”, whilst the Singaporean book focuses on 
“decolonisation” and “the construction of nation states” in Southeast Asia. 

However, the fact that different textbooks have different perspectives does not 
need to be problematic. On the contrary, in a field like history, one can discuss 
different perspectives, different values, and different ways of representing the histor- 
ical knowledge—and by doing so, deepen the historical understanding (cf. Insulander 
et al. 2016; Wheatly 2001). 

Yet another aspect we would like to emphasize here is the question of activity 
linking (Selander 2018). In the book from Singapore, we find clear links between the 
kind of activities the pupils are supposed to perform, and the kind of information, 
sources, websites, etc. that can be found in the text. In other words, the text is modeled 
on how information is to be understood and used by the reader. The pupils are also 
encouraged to work collaboratively, to think like a historian, and it is clear which 
(in this case) four questions the pupils are supposed to focus on. The Swedish book 
is much more superficial and vague. One example: “From your geography studies, 
you are used to maps that show different places. In this case, the map is used to 
describe a certain kind of event. Can you think of anything else that you can show 
with the help of maps”? Hence, instead of a question that helps the pupils to deepen 
their understanding of historical processes and events, they are here suddenly moved 
away from the topic to something totally different. 


8.6 Religion, Elementary School 


In the Scandinavian countries the subject Religion has its origin in the subject Chris- 
tianity. Today, this subject is supposed to be a guide to the stories, holy places, 
rituals, and symbols of different religions, and furthermore, to be an arena where 
ethical questions can be discussed. It might seem inevitable that in description of 
religions from a Western perspective there is a central, symbolic space—in Lotman’s 
(1990) terms, a semiosphere!—in this case with Christianity as the natural point of 
departure. This can be explained by quite divergent things, such as the dominance 
of Christianity since the Middle Ages, or the historical roots of the school subject, 
but also the political discussion. Here we will examine how different religions are 
introduced in different ways in a Swedish textbook. The differences concern things 
like what facts to present and the different illustrations chosen with regard to the 
religions and those who practice the religion in question. We will return to these 
questions under “Values” (Sect. 8.6.4). 

In the following, we will focus on the sections “Visiting a Christian home” (Sw. Pa 
besök i ett kristet hem) and “Visiting a Muslim home” (Sw. Pa besök i ett muslimskt 
hem; Figs. 8.6 and 8.7) in the textbook Upptäck Religion (Eng. Discover religion) 
(Ring and Sandin 2008), produced for elementary school years 4—6. 


'Lotman (1990) describes the “semiosphere” as symbolic spaces that express aspects such as centre 
and periphery, demarcations, and power relations. 
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Fig. 8.6 Visiting a Christian home. Upptäck Religion (Ring and Sandin 2008, pp. 12-13, with 
permission from Liber) 


På besök i ett muslimskt hem 


Fig. 8.7 Visiting a Moslem home. Upptäck Religion (Ring and Sandin 2008, pp. 72-73, with 
permission from Liber) 


This textbook is also produced in an online version, with the same layout as 
the printed version.” The online version supplements the printed version through 
resources such as Internet links, the possibility to add notes to the text, or to have an 
electronic voice reading sections of the text. When you click a button for “extra 
resources”, a page with a video film is shown. Apart from the film, the online 


>The images in this section are screen shots taken from the online version of the textbook. 
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version contains links to other electronic resources connected to the textbook. We 
will comment briefly on the content of the video films in relation to the different 
sections examined. 


8.6.1 General Structure and Setting 


The general structure is similar in both chapters, but we also note differences in 
the ways that a number of phenomena are presented. In both sections, the central 
written text is placed in the middle of the book spread, and illustrations and text 
boxes are placed around this written text. In the presentation of Christianity, we find 
a book of prayers in the upper left corner. Under the book, the Lord’s Prayer, “Our 
Father”—in a modernized version—is given in a blue-shaded text box. A large image 
to the right at the bottom of the left page shows a family saying grace before eating. 
Below that image, a yellow-toned text box explains words like “prayer” (Sw. bön) 
and “hallowed” (Sw. helgat). The text box has a heading “Word list” (Sw. ordlista) 
placed vertically to the left. Text boxes containing explanations of words are used 
throughout the textbook. Also, all the words explained in text boxes are italicized in 
the running text and the explanations are bolded. 

The right page contains two headings: “The cross is an important symbol’, and 
“Baptismal Candle” (Sw. dopljus). The cross is illustrated through hands clasped in 
prayer, holding a cross above a book (presumably the Bible). The baptismal candle 
is illustrated by two newly lit candles. Below that photo are two more explanations: 
“symbol” and “rise from the dead” (Sw. uppstå). In the rightmost lower corner is a 
yellow-toned text box “Did you know...” (Sw. Visste du att...), which explains the 
word “amen”. Two of the illustrations depict things (a book and a candle), while the 
others depict people in some kind of action (praying). 

In the corresponding section on Islam we find the heading “When the sun sets the 
fast is broken” (Sw. när solen gar ner bryts fastan) on the left page of the book spread. 
The right page contains the headings “Then it is time to pray” (Sw. Sedan dr det dags 
for bön) and “During Ramadan you are supposed to be particularly nice” (Sw. Under 
ramadan ska man vara extra snäll). The left page contains two illustrations, a photo 
of dates in a box with a hand taking one date, and a photo of “Muslim food” (Sw. 
muslimsk mat) with a text about halal in a blue-toned text box. On the right page, 
we find a photo showing a big table, and a small girl laying the table. Another photo 
depicts different kinds of prayer compasses. Here a text says “You can even use a 
mobile phone to show the direction to Mecca’. At the bottom of the page, to the right, 
there is a text box toned in yellow: “What do you think?” (Sw. Vad tycker du?) with 
the text: “During Ramadan you are supposed to avoid having disputes with others. 
Among Christians it is common to try to be especially kind at Christmas. Is it a good 
thing that you are sometimes forced to be kind?”. 

In the online version of the textbook three short video films are connected to 
the sections above. In the section about Christianity the film “Talking to God” is 
introduced through a short sequence from a ceremony in a Swedish church. After 
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that a longer sequence follows, where a woman explains her relation to praying. In 
the section about Islam, there are two films, “Religion at home” (connected to the 
left page in the book spread) and Ramadan (which is connected to the right page). 
In both films, informative sequences are mixed with short sequences with children’s 
comments, and sequences showing people praying or reading the Quran. 


8.6.2 Interaction Between Text Resources 


Different semiotic modes interact in the section about Christianity. One theme is 
prayer: the main text on the left page deals with prayer, illustrated by a book of 
prayers and the text of the Lord’s Prayer. The main image depicts a family praying 
at the table. Another theme is the cross (eternal life), sometimes used in prayer (the 
image to the right on top of the page). A third theme is the baptism (being christened) 
and a baptismal candle. The text box “Did you know...” deals with “amen” the final 
word in a prayer. Five central concepts are explained here: prayer, hallowed, symbol, 
rise, and amen. 

In the Muslim home, fasting is a central theme. The text says that you break the fast 
with a date, which is illustrated by a hand taking a date. At the bottom of the page 
“halal’”—in this case admissible food—is illustrated by plastic bags with chicken 
legs in a deep freezer. Another theme is prayer, illustrated by a prayer compass, and 
a third theme is the fact that you are supposed to be kind during Ramadan, which 
is compared to a Christian rule for Christmas. Three words are explained: fasting, 
prophet, and role model. There is no “Did you know...” text box in this section, but 
instead the reader is invited to ponder over calls for kindness. 


8.6.3 Figurative Language 


In the Christian home, the candle is a symbol of the transition from darkness to 
light, which in this case means the transition from not being Christian to becoming 
a Christian. The word ‘symbol’ is explained as something which is “used as a sign 
for something”, and it is exemplified by a heart, which is “as symbol of love, to care 
about”. 

The text about Islam states that you cannot eat while the sun is up, and that you 
break the fast when the sun sets, then eating a couple of dates and drinking a few sips 
of water. The text also says that the reason why this is done is because Muhammad 
did so, and that many Muslims want to do “what he did”. In what sense the position 
of the sun affects the fasting remains unexplained. 


92 8 Social Sciences 


8.6.4 Values 


If we once again use Lotman’s (1990) concept of “semiosphere”, we can clearly see 
the different perspectives that are used. In the Christian home we are invited to “an 
ordinary evening” (Sw. en vanlig vardagskvdll) where the family say grace before 
having dinner. Here, the center is a (Swedish?) middle-class family, acting in a shared 
religious ritual (praying). 

In the Muslim home, on the other hand, we are invited to a family celebrating 
Ramadan. In this case we (as viewers and readers) are not placed in the center, but look 
out from a distance. The text deals with fasting, and then prayer. No human beings are 
engaged in shared activities, instead the images depict individual activities: a hand 
taking a date, two arms lifting a packet of chicken from a freezer, and the young girl 
who is helping someone to lay a table. 

In the Christian home, the different actions connected to a Christian’s relation to 
God are described as a strong feeling of love. In the Muslim home, only the prayer 
ritual is described, and instead of love, the text focuses on the fact that you are 
supposed to be kind and helpful. It is worth pondering over how a Western reader 
would have perceived a description of “an ordinary day” in the Muslim home, with 
explanations of the relation to God, including a heart as a symbol for love, while 
the Christian home was described in terms of observable rituals during Christmas 
celebrations, and the expected kindness. Here we probably would not need any close 
analysis to experience where the center and periphery of the semiosphere are located. 

In the films that are available in the digital version of the textbook, we can hear the 
voices of Muslim children. However, since the films have been created in an English- 
speaking country, Swedish children are reading translations from the original, which 
creates a distance. The title of the film is “Religion at home” with a subheading 
“Many girls pray and read the Quran at home”. The film shows two sisters praying 
at home and how their mother teaches them about the Quran (while it is said that 
boys more frequently pray in the mosque among the other men, and that they go to 
Quran classes). 

A couple of other things are worth noting in the films, for instance that the speaker 
says that the girls are learning Arabic “to be able to read the Quran”. But is not the 
language a means of communicating at home or with relatives? In the film it is not 
clear whether the girls speak another language than Arabic at home (there is a Swedish 
voice-over when the children in the film talk during interviews), or if the film refers 
to the fact that the girls are learning the Arabic written language and classic Arabic, 
which differ from the spoken dialects. Of course, it might be asking too much if the 
film were to explain how the language of the Quran differs from spoken dialects, 
but the comment about learning the language contributes to maintaining a world of 
“us” and “them”. That perspective is especially noticeable from a comment about 
the Arabic written language: “before you have learnt the language it looks really 
tricky”. Does that mean that, for instance, the Swedish orthography looks less tricky 
for anyone not used to it? Also, “to know a language” (regardless of language) does 
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not imply that it is easy to read the language, even though language knowledge is a 
prerequisite for reading with understanding. 

The film titled “Ramadan” has (just as in the printed version) a sub-heading 
“During Ramadan you must not eat or drink until the sun has set”. In the film a boy 
tells us how hungry and tired he is during Ramadan, while a girl comments on the 
fact that you then will experience how it can feel when you are poor (and that it is 
easier when Ramadan falls during the winter months, when the sun is up for a shorter 
time than during summer). The film also shows that in the children’s school, there 
are two rooms where the Muslim children can pray. 

As mentioned, the film connected to the section about Christianity is introduced 
with a short sequence from a ceremony in a Swedish church. The speaker voice says 
that in the Swedish church there are many different kinds of sermons, and that this 
particular sermon includes a lot of music. This is followed by a longer sequence in 
which a Swedish woman tells the viewer about her relation to prayer. She says that 
the prayer can be different depending on whether she is on her own or if she is praying 
in a larger context, and she also says that the actual location is of no importance. 
Instead she can pray wherever she can feel the presence of God and the deeds of God, 
for example by the sea. As a whole, Christianity appears to be a flexible religion. 

If we compare the images we get through the video films, we note some clear 
features. For instance, Islam appears to be more distant for a Swedish reader than 
Christianity. The films about Islam haven’t been made in a Swedish context; instead 
films from an English-speaking context have been used, in which children’s voices 
are replaced with Swedish voices reading a translation. Also, these films focus on 
what is “different”: the Arabic orthography “looks really tricky”, and you learn the 
language to be able to read the Quran. The children in the films comment on how 
hard it can be during Ramadan, for instance when seeing “everyone else” eating. 

The video film with the Christian woman was made in Sweden and we get a 
personal image of her belief, which is a flexible way of practicing your religion. Does 
that mean that a Muslim cannot be different in different contexts, or that you cannot 
speak to your God whenever you experience the presence of God? How would the 
relation to “the unfamiliar” have been affected if Swedish Muslims had been focused 
on, or if the film had examined why you are supposed to pray at specific hours (not 
to risk forgetting your belief in God), or the importance of always remembering that 
you are lucky to have food every day? 

The fact that children of the same age as the supposed viewer are involved in the 
films about Islam could possibly diminish the distance between “us” and “them”. At 
the same time, the Muslim children in the films do not live in a Swedish context, 
which might appear misleading. People practicing Islam, as well as Christians, live 
in all parts of the world and in many cultures. Is there any specific reason why the 
Muslim children in these films live outside of Sweden? 
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Chapter 9 A) 
Language a 


9.1 English as a Foreign Language, Elementary School 


In Sweden, English is a language with high status, and digitization in particular 
has made English easily available so that many children learn English outside of 
school, what is called ‘extramural English” (Sundqvist and Sylvén 2016), through 
commercials, music, and different digital media, including online games, YouTube 
clips, etc. Furthermore, Sweden has a long tradition of subtitling films rather than 
dubbing, apart from animated cartoons for the youngest, which are often released in 
two versions, one dubbed and one subtitled. Thus, Swedish children even in early 
grades could be expected to have learned English through various out-of-school 
literacy practices (cf. Street 2003). 

In the Swedish school system, English is a compulsory subject from grade 3, 
elementary school (children aged around 9) though schools can choose to introduce 
English at any time during preschool class or in elementary school up to grade 3. 

In the following, we analyze a textbook in English for Swedish elementary school, 
grade 4 (students around 10 years of age). The same textbook has been analyzed in 
an unpublished work by Maria Nilsson (2015), who concludes that this text—like 
many other textbooks in English—to a great extent consists of “atypical” genres in 
a kind of “school English”. Some of the findings from that analysis are used here as 
well. 

In our analysis we will look closely at two spreads from the textbook Happy! 
Textbook Year 4 (Sutcliffe et al. 2009).! 


‘Exercises focusing on various aspects of English grammar are provided in a separate exercise 
book. 
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9.1.1 General Structure and Setting 


At the start of the book, the reader is presented to the Best family, living in Manchester, 
England. In the first chapter of the book, the readers learn some basic facts about 
Adam and Kate Best who are brother and sister: their favorite sport, food, and color, 
etc. 

The book is structured in a number of chapters dealing with different content with 
the aim of letting students practice various aspects of the English language, such 
as grammatical constructions or word knowledge. The texts presented in the book 
spreads shown in Figs. 9.1 and 9.2 are supposed to give the students opportunities 
to practice constructions like “are/is wearing” and plural forms of regular nouns 
(Fancy Dress Day, Fig. 9.1). The reader is not provided with information about 
what structures or content areas the student is supposed to practice in the respective 
section. Instead this is implicit. Most book spreads have the same overall structure, 
with a heading given in a red text box, and with writing combined with—usually 
drawn—illustrations. Each spread provides the reader with a glossary with Swedish 
translations of a number of words, given in text boxes separated by dotted lines. For 
each section, there is an equivalent section in an exercise book. 

What is particularly striking in this textbook is the ample use of dialogues, regard- 
less of content. The text in Fig. 9.1 is an example of this. This is a kind of dialogue 
that would hardly be heard in natural conversation; instead it gives some kind of 
description of the illustration filling the top of the book spread. Thus, instead of 


Fig. 9.1 Fancy Dress Day, Happy Grade 4 (Sutcliffe et al. 2009, pp. 34-35, with permission from 
Gleerups) 
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achieving the goals regarding dialogues indicated in the curriculum documents, the 
use of dialogues seems to be a way of structuring read-aloud activities in the class- 
room, where students can be asked to read different parts of the text (in this case 
Kate and Adam). This is a recurrent pattern right from the start of the book, where 
Adam and Kate have a dialogue presenting themselves (presumably to the intended 
reader). 


9.1.2 Interaction Between Text Resources 


As was mentioned above, each section provides the reader with a glossary. The words 
in the text boxes are presented in the same order as they appear in the text, which 
makes the words relatively easy to find. The illustrations given in each section are 
clearly connected to the writing on the same page, and they can therefore support a 
reader who is unfamiliar with the content of the text. However, the illustration of the 
rainbow shown in Fig. 9.2 can hardly support a reader who is unfamiliar with the 
physics of light. In writing, information is given about the ways in which different 
colors bend. Here the rainbow with a pot of gold gives no support for the reader 
who wants to know more about this phenomenon. This book spread is one of few 
examples with a content that is clearly connected to other subject areas—in this case 
science. Thus, here the students have an opportunity to actually learn more about 
light spectra at the same time as they practice their English in the area of science. 


The Colours of the Rainbow 


Fig. 9.2 The colors of the Rainbow, Happy Grade 4 (Sutcliffe et al. 2009, pp. 26-27, with 
permission from Gleerups) 
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Yet it is obvious that such a benefit is not the intention of the author, not least since 
it presents the golden pot at the bottom of the rainbow as it is in fact for real. 


9.1.3 Figurative Language 


There is no figurative language in the sections we use as examples of this textbook, 
apart from Adam’s comments that the jugglers look like parrots and Kate’s comment 
implying that Adam is “a clown”. Given the difficulties language learners can have 
with figurative language, the lack of such expressions is more or less expected in a 
basic language textbook like this. 


9.1.4 Values 


The textbook as a whole reveals a number of implicit values. Examples are the 
traditional roles that boys and girls are given in the illustration in the text about the 
Fancy Dress Day. The illustration takes up the description of the only girl in the text 
as having dancing lessons and wearing a pink skirt and silver tights. In the illustration 
she poses in ballet fashion. In writing she is also said to be an acrobat, but that is 
not shown in the illustration. The boys, on the other hand, form a band and in the 
illustration they are remarkably active in their playing. 

In other parts of the book, English people and their habits are described in quite 
stereotypical ways. In one section it is said that “In England many people have a 
full English breakfast”, and that “Many people have a special Sunday lunch of roast 
beef, Yorkshire pudding”, etc. 

Less traditional is the description of the elderly in an overall section called 
“Friends”. Here a band is presented, consisting of residents in a retirement home. 
They all seem to be quite healthy and active, for instance forming a band and being 
interested in motorbikes and the like. 


9.2 English, Secondary School 


Our second example regarding languages is taken from the Singaporean textbook 
All About English (Doyle et al. 2011). Thus, this example does not concern foreign 
language learning, although in Singapore, English is one of four official languages 
(the others are Mandarin, Tamil, and Malay). Therefore, Singaporean students—as 
well as students from other linguistic backgrounds—can be expected to not to have 
English as their mothertongue or strongest language. The textbook has four main 
sections: Listening and viewing, Reading and viewing, Speaking and representing, 
and Writing and representing, thus giving attention to meaning-making from reading 
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Why We Need Good 


Pronunciation 


proving 
Pronunciation 


What are Consonant 
Clusters? 


Fig. 9.3 Improving pronunciation, All About English, secondary school (Koh et al. 2011, pp. 104— 
105, Reproduced by permission of Hodder Education, photographs blurred for copyright reasons) 


and creating texts, both paper-based and digital, including films, etc. Each section 
consists of a number of chapters. In the following, we will have a closer look at one 
chapter in the section on Speaking and representing: “Improving pronunciation”. 
Figure 9.3 shows the first two pages of the chapter. In the following, we concentrate 
mainly on the general structure and values. However, since multimodality is the 
focus of our book, it is of interest to take a brief look at how multimodality as a 
theme connected to work with text is handled in this textbook. The textbook—just 
like the Singaporean textbook in science that was used as an example to present 
our model in Sect. 4.5—is in itself complex regarding the use of different visual 
representations (see Figs. 9.3 and 9.4). The layout is largely built on a variety of 
visual resources, such as photographs, text boxes, tables, and color, and recurring 
symbols indicate connections with, for instance, preceding chapters or textbooks or 
aspects of the content that the students are supposed to pay special attention to. But 
how is multimodality as a phenomenon treated in the text? First, the chapters dealing 
with speaking and writing are denoted “Speaking and representing” and “Writing 
and representing” (our italics) respectively, implying a possible multimodal focus 
on communication. However, a closer look at these chapters reveal a limited focus 
on other resources than speech or writing. Some comments are given on layout and 
the use of images in the chapter on writing, though mainly in a very general sense, 
for instance that photographs can be included in a travel report. In a section on 
how to write informative brochures some comments on layout are given, though 
commonly quite vaguely, like “Choose a layout and organization that is suitable for 
the information you are giving and that will help to achieve your purpose”. However, 
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Fig. 9.4 Improving pronunciation, All About English, secondary school (Koh et al. 2011, pp. 106- 
107, Reproduced by permission of Hodder Education) 


little is said about how to do that. Now let us take a closer look at one chapter in the 
section Speaking and Representing, namely Improving Pronunciation (Figs. 9.3 and 
9.4). 


9.2.1 General Structure 


We can note that the chapter starts off by giving an everyday example of what is 
viewed as a common mispronunciation in the vernacular Singaporean variety of 
English: a boy, Ravi, pronounces “theme” as “team” which results in a misunder- 
standing. The reader is told that you need to have “good pronunciation” to avoid 
similar situations. Then the reader is given an introduction to pronunciation together 
with more arguments for the importance of correct pronunciation. The rest of the 
chapter has a thematic structure, first dealing with consonant clusters and then vowel 
sounds, including diphthongs. In these sections the reader is given overviews of 
the English sounds, combined with instructions about how to pronounce the sounds 
correctly, with examples of common mispronunciations to avoid. The International 
Phonetic Alphabet (IPA) is used, and symbols indicate that audio files are available 
through digital resources (in this case, a compact disc). 

The text includes a variety of textual resources. Apart from printed words in 
headings, text-box-like sections labeled “Connection”, and running text, the first 
introductory book spread contains a comic strip with school children and photographs 
of professionals working in areas where good pronunciation is important. In the 
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succeeding pages, tables are commonly used to present English words with similar 
spelling or pronunciation in systematic ways, presumably based on sounds where 
the pronunciation in vernacular Singaporean English differs from Standard English. 
Throughout the book, symbolic images like a shackle (p. 105) for connections to 
another book in the same series or an image of a dictionary for definitions (p. 106). 

Just as in the example from the Singaporean science textbook that we used to intro- 
duce our analytical model, the starting point of the chapter is an everyday problem 
followed by a movement towards systematic descriptions similar to the ones used 
in the discipline, in this case linguistics. In both of these textbooks, the movement 
towards the discipline concerns both content and text resources used: the comic strip 
and photos of people using spoken language on the first two pages, followed by 
tables containing subject-specific symbols from the IPA, with terminology such as 
‘syllables’, “consonant clusters’, ‘suffix, and ‘phonemes’ in the running text. These 
terms are all explained in the text. 


9.2.2 Interaction Between Text Resources 


Regarding the interaction between different text resources used, this chapter involves 
no apparent challenges. Images are placed close to the running text they refer to, for 
instance the two professionals who are placed to the right of the column about which 
professions involving a lot of verbal communication, even though it might not be 
completely clear what professional role the male sitting on a stool has. The tables with 
subject-specific symbols are integrated in the running text, with explicit connectors 
such as “the following tables show...”. 


9.2.3 Values 


The theme chosen for this chapter clearly reveals underlying values concerning “right 
and wrong”. A main point is that there is “good” and “bad” pronunciation, and 
that the Singaporean variety of English is not “good pronunciation”. As mentioned, 
the starting point is a made-up problem, with a misunderstanding caused by a /th/ 
pronounced as /t/in line with vernacular Singaporean pronunciation rules. The reader 
is told that you need “good pronunciation” for two reasons. First to avoid misunder- 
standings. Second to make sure that the listener “pays attention” to what is said. The 
second reason is clearly value-laden, implying that vernacular pronunciation—fully 
legitimate within the Singaporean community—would reveal you to be less culti- 
vated, and that you would therefore be less inclined to be taken seriously by people 
who “count”. Thus, this statement implicitly contains values about good and bad 
language. The next two pages, though, are explicit about what the student should 
and should not do as regards pronunciation, and here, all examples given are “mis- 
pronounced words” in Singaporean English. Such a normative perspective can be 
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contrasted to the ways dialects and sociolects are treated as functional varieties of a 
language in much of modern sociolinguistics (e.g. Wardaugh 2006). 
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Chapter 10 R) 
Subjects Focusing on Practical Work crest 


In this section, we use two Swedish examples from subjects focusing on practical 
work, one from textile crafts in lower secondary school, and one text explaining 
welding techniques in vocational studies (the Industry program) in upper secondary 
school. 

Texts connected to subjects involving craft techniques are specifically interesting 
from yet another multimodal perspective: instruction for action. In the Swedish 
school context, such subjects can be found at all levels of the school system, in 
subjects like crafts (textile crafts or carpentry and metal crafts), as well as vocational 
programs in upper secondary school oriented towards industry or automotive engi- 
neering. In many cases those subjects are not perceived as theoretical, while instead 
the “doing” is in focus, and the texts used in these subjects would then function 
as instructions (Ekström 2010). Thus, the text becomes essential for the doing, and 
many illustrations carry important information. However, it is not always the case 
that the illustrations actually function as instructions, something which will become 
evident in the analysis of the text from welding technique below. 


10.1 Crafts, Secondary School 


The text we use here is one that was used in a textile crafts classroom which was 
investigated for its literacy activities by Eva Lindqvist (2014). In her description of 
the classroom activities it was evident that the students used a variety of texts. No 
textbook was used, but instead different kinds of compilations or texts produced by the 
teacher. Regarding sewing or knitting instructions, various descriptions of the process 
produced for education were used. When there were no pedagogic texts available for 
certain processes, the students were referred to quite complex instructions available 
from the Internet. The text that we use here consists of two pages from a three-page 
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FÖRKLÄDE 


Rita av monsterdelama: förkläde 
ficka (2 st) 
knytband (2 st) 
nackband 


Klipp ut och nåla fast pa tyg 
ä ån ingår som mönstret anger. 


VISA. Klipp ut / 


; 


Vik en dubbel fåll på bröstlappens svängda sidor. 
Längd-nåla / 


VISA 


Sy raksöm 2 mm fran inner-kanten. 
S! Ta ut nålarna efter hand du syr. 


Vik en dubbel fall pa sidorna, först ca: 1 cm och sedan 1% cm. Nala 
VISA 


Sy 2 mm fran innerkanten 


Vik 1 cm pa ovan-kanten mot avigsidan 
Pressa med strykjam 


Vik ner kanten 3 cm. Tvamala. 
VISA 
Sy raks6m 2 mm fran innerkanten. 


Vik och sy neder-kanten (=fallen) 
pa samma satt 


Fickor 

e Vik ner ovan-kanten pa fickorna 2 gånger - 
först 1 cm och sedan 2 cm. Tvamala 
VISA 


Sy raks6m 2 mm fran inner-kanten 


Fig. 10.1 Instructions for sewing an apron, p. 1. Secondary school (re-printed with permission 
from Kerstin Juhlin) 


booklet produced for teaching textile craft.! It was used in the studied classroom as 
an instruction for those who chose to make an apron (Figs. 10.1 and 10.2). The third 
page—which is omitted here—is a drawn pattern which the student is supposed to 
enlarge and transform on to a piece of paper to use when cutting the fabric. 


' An experienced teacher in textile crafts has produced the instruction and made it available on the 
Internet (the text is now taken away from the web page). 
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Sick-sacka fickornas traskanter. 
Vik in 1 cm runt fickorna och pressa lätt. 


Placera fickoma på förklädet och tvärnåla fast dem 
VISA 


Sy fast fickorna med raksöm % cm från kanten. 
Fäst i början och slutet, vid ficköppningen 


Knytband och nackband 


e Vik in långsidoma 1 cm på knytband och nackband. 
Pressa lätt 


Vik därefter banden med långsidorna mot varandra 
och tvamala 

VISA 

Vik även in en av ändarna pa knytbanden. 


Sy raksöm ca: 2 mm fran kanten 


Nala fast banden pa forkladet och prova. 
VISA 
Sy fast med raksém Se skiss 


Fig. 10.2 Instructions for sewing an apron, p. 2. Secondary school (re-printed with permission 
from Kerstin Juhlin) 


10.1.1 General Structure and Setting 


The writing is structured in bullets, and placed to the left on both pages, with headings 
in italics. To the right are drawn illustrations with no headings. Both the writing and 
illustrations are visually prominent. The stylization of the illustrations, with clearly 
marked contours in black, contribute to their prominence on the page. 

The expected reading order might be to follow the bullets one by one and shift 
between writing and illustrations. 

Apart from the first heading, “Apron” (Sw. Förkläde), which is centered and set 
in a bigger font, the headings correspond to different parts of the apron. The first part 
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of the process—sewing the main part of the apron—has no heading. This instruction 
appears directly under the main heading. 


10.1.2 Interaction Between Text Resources 


The illustrations are central in the description of the process. The stylization 
contributes to the “doing” that is expected from the text: seams, hems, etc. become 
fairly distinct. The written text refers to the illustrations, though not explicitly, as 
might have been expected in an instructive text like this. 

Furthermore, a number of concrete, subject-specific concepts are used only in 
writing and not in the illustrations. Thus, it is implied that the student either is familiar 
with the concept, or that a teacher is expected to support the process. Quite often you 
can guess what the term refers to, for example to pin lengthwise (Sw. längdnåla) or 
crosswise (Sw. tvdrndla) on the first page. However, there are also instances where 
there is nothing in the illustration that corresponds to the written instructions. One 
such example can be found on the second page, where in writing it is said that you 
should “zigzag the raw edges of the pockets” (Sw. sick-sacka fickornas traskanter). 
Neither “raw edges” nor zigzag stitches are shown in the illustrations. Instead, the 
illustration connected to this part of the instruction depicts the next stage, namely 
the folded edges along the sides of the pocket. Therefore, what corresponds to “raw 
edges” or the result of a zigzag seam is expected already to be clear for the student 
using these instructions. 

The various illustrations lack headings or captions. Thus, the reader is expected 
to know when to use the illustrations and when only writing is expected to carry 
information about the process. 

The illustrations function mainly to depict various parts of the process, or to 
visualize or concretize what is said in writing. One example is where it is said in 
writing that the edges of the curved part of the apron should be pinned lengthwise, 
and where the illustration depicts pins along a hem. 

At times, the illustrations supplement the writing, for example when the illustra- 
tions only depict the “reverse” (Sw. avigsidan) or at the bottom of the second page, 
where the text says “Sew together using straight stitching. See illustration.” This is 
the only place where the text explicitly refers to an illustration, although which one 
of them is implicit. What is probably referred to is the detailed illustration of the bib, 
where a cross marks the place where the tying strips are supposed to be stitched to 
the apron. 


10.1.3 Values 


There are no explicitly expressed values in this text. However, we may note an 
implicit discourse based on meticulousness and control. The expected accuracy is 
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shown through the detailed descriptions of the process (exact distances between edge 
and seam, and instructions as to how to fold the hem). The same thing is shown in the 
illustrations depicting details in the process. The importance of control is shown in 
the recurring request “SHOW?” (Sw. visa) in the writing. Thus, the teacher is supposed 
to check that central steps during the process have been performed correctly—and 
probably in a meticulous way—before the student is allowed to go on to the next 
step. 

With a greater focus on creativity, the text might have included some encour- 
agement to use your imagination regarding the choice of fabric, or perhaps urged 
students to make decisions regarding ways of designing different parts of the apron. 
Of course, nothing prevents the teacher from encouraging the students to be creative 
in the classroom practice, but if we only look at what the text encourages, meticu- 
lousness and control appear to be important. The same thing was noted in the welding 
text, which we will look at more closely in the following. 


10.2 Welding, Industry Program 


In vocational studies in Swedish upper secondary school a number of subjects 
involving practical processes are included. The Industry program includes a number 
of practical courses with direct relevance for future work in industry. However, to be 
able to pass such courses, a knowledge of subjects like physics is also required. In this 
particular case we will look closely at a course in welding techniques. According 
to the curriculum, students are supposed to develop knowledge about the risk of 
deviations in form and inaccuracies, and how to avoid and remedy the faults. 

In a textbook on welding (Henriksson and Hallman 1999) one chapter deals weld- 
ability (Sw. “Svetsbarhet’’). This covers aspects such as how shrinking and expansion 
in the metals are associated with temperatures, and how this has consequences for 
the welding process (Henriksson and Hallman 1999, pp. 95ff). 

In a doctoral thesis, Pia Visén investigated text talk in upper secondary school. 
In her investigation, this section in the textbook was used (Visén 2015, also see 
Hallesson and Visén 2018 in which text talk during the same lesson is analysed). In 
her study she showed how the teacher in his exposition about welding stuck very 
closely to the textbook, though never referring to any sections in it explicitly. Her 
analysis also revealed that the teacher made no connections to the illustrations in the 
textbook. 


10.2.1 General Structure and Setting 


The book spread contains both writing and illustrations such as diagrams. The writing 
is sequenced in sections under a number of headings. The illustrations are placed 
either to the right of the written text or below it. Both writing and illustrations are 
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OM AON 


vissa fall räcker det in 


semalisering, och avspänning kallar ban dirf 


frit. Efrnom materialet krymp 


håll uppstår spunnlagar längs + 
I auteriallera för svetsare beskriva seter 
atburgta | detal ning 


Otvidgviag och kryrupsiag 


ning mirsa n stång av 


z at det blir baktig: eller vågkrr 
stål blir 9 men linge per meter rä 


vårkryripningen inte så stor inverkan 


ha 
der varas sd att iomperstuser her 
600 °C. Nir den sval zá På actvtsstycken 
sil sin unprunglåga I 


När man svetsar blir materia 


k 
et uppvårmt till en tempera t brakar Bland kalim “Lowrenga 
ur som medför en betydande säger också art materialet “slår seg’ 
anpdiduurg, Nir neben 


ni n i svebens tjock 
ksnkterag sker en vas 
irympnirg 


Arympspoaningar 

Om detaljen som svetsas år fast Inspåed | Lorserak 
tienen så alt materialet inte kan krympa feet uppstår 
kvarstiende spånningar | ogynesarens fall kan spèr» 
ingar w Må så stora att materalet bester 


Fig. 10.3 Welds. Facts about welds. Bågsvetsning. Fakta om bågsvetsning (Henriksson and 
Hallman 1999, pp. 96-97, with permission from Liber) 


prominent, and the placement of the illustrations invites the reader to read from the 
top of the text and then down, shifting between writing and illustrations (Fig. 10.3). 

The content is thematically arranged, starting with overarching content about how 
materials are affected by heat and cold. The headings in the book spread, “Expan- 
sion and shrinkage” (Sw. Utvidgning och krympning), “Tensions due to shrinkage” 
(Sw. Krympspdnningar), and “The effect of shrinkage” (Sw. Krympningens verkan) 
support the reader in following the thematic structure. 


10.2.2 Interaction Between Text Resources 


The book spread contains different types of illustrations. One example is the 
schematic image, or sketch, of a weld on top of the left page. This illustration is 
more abstract than the other illustrations, and it is of a type that can be expected to 
be found in handbooks for welders, that is, for people working in the profession. The 
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rest of the illustrations concretize the content given through writing, with a design 
that appears to be more in line with concretizing images in instructional pedagogical 
texts. Writing and illustrations are closely connected and the illustrations cannot be 
fully understood without the written information, while the illustrations concretize 
or complement the information given through writing. 

Throughout the text, the images consist of concretizing images focusing on the 
weld and the material connected by the weld, which is fully congruent with the 
writing. Metal pieces are depicted in blue color while the weld is purple. It is partic- 
ularly instructive that the more abstract depiction at the top of the page and the more 
concrete illustration at the bottom of the same page exhibit clear resemblances. 

Thus, there is a clear congruence between illustrations and writing in this text. 
Further, the illustrations are closely connected to the writing on the whole. Also, 
regarding the layout of the book spread, illustrations are placed in direct relation to 
the written text. Moreover, the same concepts are used in both writing and illustra- 
tions, with one exception. This concerns the sketch at the top of the page, depicting a 
cross-section of a weld, presenting a number of terms. These terms are both subject- 
specific (e.g. “Weld metal” [Svetsgods], “Weld face” [toppyta], “Weld Junction” 
[smdltgrédns]) and more general concepts (““Heat-affected material” [vairmepdverkat 
material], “Base material” [grundmaterial]). None of these terms are mentioned in 
the corresponding written text. The same is true of the abbreviation HAZ, which 
stands for Heat Affected Zone,” an abbreviation which is not explained in the book. 
The meaning of the abbreviation is implicit in the abstract sketch, since the abbre- 
viation is placed between brackets after “Heat-affected material”. Later in the book 
(p. 139), the abbreviation is used in writing, although here too without an expla- 
nation: “In order to test the malleability of the weld and the area around the weld 
(HAZ) a bending test is made.” In the textbook, the abbreviation is used in relation 
to both “heat-affected material” and “the area around the weld”. In the latter case it 
is implicit that HAZ is connected to an area affected by heat (i.e. the area directly 
connected to the weld). 

In relation to the image at the center of the left page, we note the conventionalized 
color coding for cold (blue) and heat (red). A welding nozzle directed towards the 
red bar also functions as a way of elucidating the heat. 

The illustration in the middle of the left page is interesting in a number of ways. 
It is supposed to elucidate that metal expands when heated. According to the illus- 
tration, this means 9 mm per meter, if steal is heated to 600 degrees Celsius. For 
obvious reasons, the distances in the illustrations are fallacious. At the same time, 
the convention of marking exact distances between the ends of two arrows is used, as 
in the case of the meter length. In technical drawings, however, the relations between 
different dimensions or distances are usually scale models with conventions such as 
a broken or dashed line to mark broken lines, which could have been utilized in this 
text too, to elucidate that the figure is not drawn to scale. 


*In the text talk that Pia Visén (2015) analyses, the teacher explains this concept using both the 
English term and a translation into Swedish (varmepdverkat material). 
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Moreover, the illustration contains a mixture of concretion and abstraction when 
depicting a welding nozzle heating the steel at the same time as conventionalized 
choices of red and blue for heat and cold are used. However, it is unlikely that the 
welding nozzle could heat a whole meter of steel up to 600 degrees Celsius. 

The illustrations contain arrows denoting different things. In the uppermost illus- 
tration, arrows are used both to mark the labeling of different parts of a weld seam 
(“Weld metal” [svetsgods], “Weld face (reinforcement)” [toppyta (rage)], ““Penetra- 
tion bead” [Rotvulst], “Weld junction” [smältgräns]), and to mark a limit between 
the material directly connected to the weld “Heat-affected material [HAZ]’ and the 
unaffected material. Here two arrows pointing two ways from the label are used. 
There appears to be no reason why no arrows have been used to mark “Base mate- 
rial” and “Unaffected base metal” (Sw. Opdverkad grundmetall). From the labeling 
no information is given as to whether the material on the other side of the weld is 
“base material” or something else. One may also wonder whether there is a difference 
between “metal” and “material” in the illustration. 

Another way of using arrows can be noted in the middle of the page. Here the 
upper arrow, pointing two ways, is used in accordance with the convention of marking 
the beginning and end of a length, in this case 1000 mm (representing the metal 
before being heated). For some reason the same kind of arrow is not used to mark 
beginning and end of the 9 mm expansion that occurs during heating. Instead, that 
arrow points to the right line demarcating the end point for the metal balk during 
heating. Another way of showing the expansion, which would be more in line with 
the genre convention, would be to write 1009 mm at the red (hot) girder, by analogy 
with the length mark for the blue (cold) girder. 

In the illustration at the bottom of the page, the arrows mark shrinkage. Here a 
number of arrows are used in different parts of the illustration. Some of them are 
placed in the depiction of the weld while some of them are placed in the different 
parts that are supposed to be connected by the weld. 

Throughout this textbook, illustrations contain important information at the same 
time as they are quite demanding for the reader. The written text is demanding too, 
due to a style associated with the language of science. Examples of challenges in 
the language of science are a large number of subject-specific terms (weld junction, 
HAZ) and the use of so-called grammatical metaphors, such as converting verbal 
phrases into noun phrases (e.g. Schleppegrell 2004). One such example in this text 
is “a considerable length increase” (Sw. en betydande längdökning) (cf. “the steel 
girder becomes much longer”). One effect of such conversions of verbal phrases into 
noun phrases is that the text becomes denser. At the same time processes (something 
becomes longer) are transformed into phenomena (increase [noun]). Such linguistic 
choices have been observed to be challenging for students, and it is claimed that 
students might need help to “unpack” these grammatical metaphors (e.g. Fang and 
Schleppegrell 2008; Halliday 1998/2004). It is common for teachers to be aware that 
subject-specific concepts can be challenging, though the awareness of grammatical 
metaphors is less widespread among teachers. 


10.2 Welding, Industry Program 111 


10.2.3 Figurative Language 


This text contains metaphors that are part of the language of technology or science, 
such as tensions (Sw. spänningar) and stress relieving (Sw. avspänningsglödning). 
We also note that the authors of the book make this explicit in a case where the 
metaphor about the way that a material can change its shape may be mistakenly 
understood as an everyday expression. In this case, the expression results in a homo- 
phone which means “warp” but can be understood as the colloquial phrase meaning 
“hurt oneself’ (Sw. slå sig). The quotation marks and the words “we say” indicate 
that there is a the potential ambiguity in this expression, however. 


10.2.4 Values 


There are no explicit values in this text. However, as in the crafts text above, there is 
a clear message that within this discipline you need to be both knowledgeable and 
meticulous to avoid inferior results when welding. 


10.2.5 Classroom Focus 


This text functions as a basis for doing something (joining pieces of metal by welding) 
at the same time as it is highly theoretical. Also, the content is a prerequisite for the 
high quality of the final product (the weld). Taken together, in particular the scrutiny 
concerning the interaction between text resources reveals that this is a complex text, 
as regards both writing and illustrations. For such texts, text talk and mutual reading 
can be efficient for supporting the students in their text use. Hence, concerning texts 
like this, we would strongly encourage teachers to first make a close reading of 
the text to uncover possible challenges concerning the text resources used and the 
interplay between them. As a next step, the teacher and the students discuss the text, 
focusing on the different text resources and to clearly connect the discussion to the 
content and how to grapple with texts like this. 
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Chapter 11 A) 
Mathematics E 


Here we have chosen to analyse a section from The Cambridge Primary Mathematics, 
Learner’s Book 2 (Moseley and Reese 2014). It is an ambitious textbook launched 
as “a supplementary resource that consolidates and reinforces mathematical learning 
alongside the Cambridge Primary Mathematics Teacher's Resource 2”. At the bottom 
of each page there is information that relates to a “core activity”. The resource 
is “dedicated to helping schools develop learners who are confident, responsible, 
reflective, innovative and engaged” (Introduction page). In the following we shall 
here look closely at two pages, the one about Patchwork, the other about Balls, and 
in this, we focus on the interaction between text resources used. 


11.1 Interaction Between Text Resources 


At the top of the page about Patchwork there is text that directly leads the reader into 
the visual illustrations (“Look at the pictures”), followed by the questions: “What 
can you see?” and “Talk about what you can see.” Here we can immediately see the 
intentions behind the book: look, reflect, think, and talk about it. Fig. 11.1. 

The following text concentrates on squares and triangles, of the same or different 
size. However, by just looking at the page, it is not evident what the purpose is, and 
this shows clearly enough that the book is to be seen as a complementary resource 
to the teacher’s introduction and instructions. 

With this in mind, let us have a closer look at the text. In the upper left illustration 
we notice squares with lines, waves, and dots. The illustration in the middle is focused 
on squares and triangles, while the one to the right also contains some kind of extra 
information in the form of wave-like patterns (fishes or eels with eyes?). This extra 
information (lines, waves, dots, and patterns) might distract the focus on squares and 
triangles. From the book alone, then, it is not clear that you as a pupil should train to 
make comparisons between different geometrical forms. It also seems unclear what 


© The Author(s) 2021 113 
K. Danielsson and S. Selander, Multimodal Texts in Disciplinary Education, 
https://doi.org/10.1007/978-3-030-63960-0_11 


114 11 Mathematics 


Patchwork 


Look at the pictures. 


HH 
4 


What can you see? 
Talk about what you can see. 


What other patterns can you 
make using squares and triangles 
with a right angle? 

What if all of the triangles are 
not the same size? 


Fig. 11.1 Patchwork. Cambridge Primary Mathematics Learner's Book 2 (Moseley and Reese 
2014, p. 72, with permission from Cambridge University Press) 


to do with this information, since there is no explicit instruction that asks you to do 
what the two pupils (at the middle/right illustration) are doing. 

The following page about Balls seems to be even more complicated (Fig. 11.2). 

This page starts with a “block graph”, under which we find a basket with balls 
of different sizes and colors. These illustrations are then accompanied by a number 
of questions. But let us start with the illustrations. What might be a bit confusing 
is that there are no clear links between the colors of the bars in the graph and the 
colors of the balls you can see. So, after a while you perhaps think that the graphs 
are not about how many balls there are of each sort, even though the word “balls” 
under the graph may indicate this. Color is also a textual resource, and thus carries 
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Look at the block graph. 


1 block = 10cm 


balls 
soft bal 
tennis Which type of ball do you think is 
shown by the red bar? Why? 
What about the other colours? 
Which ball bounces the highest and 
500c which ball bounces the least? 


What if you had a super ball that could bounce the 
same as all of the heights added together? 
How high would that bounce be? 


What if you used only big balls? 

Would they always bounce to the same height as each other? 
What if you used only small balls? 

Would they bounce to the same height as each other? 

How do you know? 

Why do some balls bounce higher than others? 


Fig. 11.2 Balls. Cambridge Primary Mathematics Learner’s Book 2 (Moseley and Reese 2014, 
p. 72, with permission from Cambridge University Press) 


information. In this specific case, however, it might be confusing since you cannot 
see all the balls in the basket to which the graph refers: one light red ball (not four), 
one green ball (not five) two purple balls (not three), but two yellow ones (as in the 
graph). 

One of the following questions is: “Which type of ball do you think is shown 
by the red bar?” You may even wonder if this question is linked to the following 
one: “Which ball bounces the highest and which ball bounces the least?” With these 
questions we have left the mathematical definitions, and the textbook here refers to 
the reader’s own (culturally based) empirical experience or knowledge. If you do not 
have this knowledge, you are left to guesses only. 

In sum, we could say that a rather reduced and focused text like this one contains a 
number of possible challenges. It therefore seems important that the teacher discusses 
some of the missing linkages between the written text and the visual illustrations, 
makes clear what the task is about and perhaps also introduces some experimental 
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work, so that the pupils instead of guessing could explore different qualities of 
different balls, and make up new graphs themselves. 


Textbooks 


Moseley, C. & Rees, J. (2014). Cambridge Primary Mathematics. Learner’s book, 2. Cambridge: 
Cambridge University Press. 
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Chapter 12 A) 
Websites as Learning Resources get 
for Young Learners 


Today there is a plethora of Internet sites aimed at young learners. In the following we 
comment on a website, National Graphic Kids, published by National Geographic 
as an example. However, due to copyright reasons, the figures used in the following 
derive from a similar Swedish website, Youngfacts. The interested reader is referred 
to the following link to get access to National Geographic Kids: https://www.natgeo 
kids.com/uk/category/discover/. We will not go through all aspects of the model, but 
instead give some general comments on that type of learning resource. 

Figure 12.1 gives an image of a common overall layout of such websites. At the 
general level, they typically invite the user to explore the content independently, by 
using links given through images combined with written words. 

In both National Geographic Kids (NGK) and the Swedish example, the starting 
page contains links through images inviting the reader to explore a number of “facts” 
related to, for instance, themes as science and history. Examples from NGK are a 
cartoon-like image of a woman combined with “The life of Rosa Park” in words, 
and an image of viruses combined with “What is coronavirus?”. Through menus at 
the top of the page of NGK, the user can choose to focus on one theme at a time, in 
this case Animals, Science, History, and Geography. Here we note that Animals is a 
category separate from science, probably since visitors to the site may be expected to 
be especially interested in animals. Also, that page calls for some interactivity, with 
a quiz based on multiple-choice questions, though for the most part, links lead to 
informative sub-pages with texts to read, just like the Swedish website (e.g. Fig. 12.2). 

As regards interpersonal aspects, it is obvious that the site has two expected 
audiences: the curious child and an adult (teacher or parent). The voice is direct, 
addressing the child as “you”, and there is an extensive use of exclamations marks. 
Furthermore, facts are presented as “fun”, “cool”, or “amazing”, in ways that are 
not expected in traditional textbooks. Also, colloquial expressions such as “ready to 
blast off?”, or “this fab fruit?” (the tomato) are used, further enhancing the idea that 
learning is really fun. The images providing links are often based on indisputable facts 
rather than more complex problems, for instance “10 facts about. ..”. The general tone 
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ann få 
n da bidra, 


nast besvarade 
å anelen 


Fig. 12.1 Screenshot 1 from website for young learners (UngaFakta 2020, printed with permission 
from UngaFakta) 


is that it is fun to learn: from their website young learners “discover some incredible 
secrets, awesome theories and explore some bonkers experiments”. 

When clicking the links, the visual presentation of the texts is similar to that 
of textbook texts with images combined with writing (see Fig. 12.2). In NGK, the 
language style is a combination of colloquial expressions and specialized termi- 
nology, for instance in a text about the digestive system where expressions like “the 
chewed grub” are combined with “salivary glands” or “oesophagus” (https://www. 
natgeokids.com/uk/discover/science/general-science/your-digestive-system/). 

To sum up, such websites display a balancing act with a prospect to draw the child 
into traditional “school content” as something fun. This is done by the use of clickable 
images leading to “Facts about ...”, often with photos and expressions displaying 
an overuse of exclamation marks and adjectives such as “awesome” and “exciting”, 
often in combination: “10 fabulous facts about Diwali!”. When clicking the links, a 
textbook like page appears presenting indisputable facts through the combination of 
colloquial style and a specialized language. 
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Tidsperioder | Viktiga personer | Samhälle 


Tidsperioder 


Landet Egypten har en mycket lång och spännande historia. Det 
fomtida Egypten sträckte sig från Nildalen fram till Medelhavets 
flodmynningar. Men den egyptiska kulturen bredde ut sig längre än 
så, till exempel till länder som Nubien och Sudan. Det forntida 
Egyptens historia brukar delas upp i tre stora tidsperioder kallade 
”riken”. Mellan dessa faller andra perioder. Historien delas också 
upp efter 30 dynastier, de kungliga familjer som styrde landet. 
Senare kommer antikens Egypten, då Alexander den store erövrat 
landet och kungligheter med ursprung i Makedonien blev härskare 
under lång tid. 


På denna sida används förkortningen f.v.t , vilket betyder "före 
Västerländsk tideräkning” och v.t. som betyder "västerländsk 
tideräkning”. Dessa förkortningar ersätter "före Kristus" och "efter 
Kristus”, men utgår från samma sorts historiskt tidsbestämmande. 


a> 42 € 


Fig. 12.2 Example of text provided behind links: Egypt of the Pharaos (UngaFakta 2020, printed 
with permission from UngaFakta) 


Textbooks 


Ungafakta (2020) www.ungafakta.se [2020-11-18] 
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Chapter 13 A) 
Educational Games and Online E 
Resources 


The importance of games and simulations within education are increasing (Arnseth 
et al. 2019; Gallagher 2015; Mørch and Thomassen 2016; Squire 2011). In the 
following we have analysed one example of a game developed for educational 
purposes, namely The Economy Game. It is not a new one, and it appears as quite 
traditional. Then we will comment on the Swedish game Minecraft. 

In applying our analytical model to games and other online resources, including 
such resources as the website commented on in Chap. 12, we are well aware of the 
difficulties related to multi-layered and dynamic representations, where the user's 
actions in many ways will steer which information will be brought to the fore and 
seen on the screen. At the same time, the architecture behind a game does not allow 
for any kind of actions, and, as we shall see, there might also be certain values 
embedded in a game. 

We here want to emphasize three things: (1) games must be understood as a text 
that relates to the actions of the player, not as a fixed text; (2) therefore, the best way to 
study this type of text is to study gaming; (3) it is possible, however, to capture some 
salient aspect of a multimodal text of this kind, as well as some of the underlying 
values. 


13.1 The Economy Game 


This learning game is freely available from a website called “Young Facts” (Sw. 
Unga Fakta), where one section is labeled “Young Economy”. In this section, the 
starting page tells the user that it is important to have some knowledge about one’s 
own economy. The starting page of the game is shown in Fig. 13.1 and it reveals that 
there are three levels to the game. You are recommended to start at the first level, 
where you are supposed to pretend that you are going to live on your own for one 
year at upper secondary school. 
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Upper secondary 
University level 
Worklife 


Upper secondary 

Hi and welcome to the first 
4 level of the game. Here you 
—T will try what it is like to 
live in a flat completely on 
your own. You can choose 


Gymnasie +e 
Haj och välkommen till spelets första nivå. another level by clicking 
Har kommer du få prova på hur det är att bo I en alldeles egen lag one of the icons above. 
Du kommer också att få ta hand om dina egna pengar, ko m Through this level you will 
Du kan välja en annan nivå genom att klicka på en av vanto learn 
I den här nivån kommer du bland annat att få lära dig 
At PAEA BSE how to plan a budget 
PRS - to save money ona 
Att möblera din lägenhet monthly basis 
Att ta hand om dina inkomster och utg - to furnish your flat 


- to take care of your 
incomes and 


Fig. 13.1 Starting page Economy Game (UngaFakta 2020, printed with permission from 
UngaFakta) 


13.1.1 General Structure and Setting 


The structure of this game is fixed and the layout is relatively sparse. When starting 
the game you first have to go through a number of informative pages about how to 
play the game, for instance that you first of all need to make a budget. Here you also 
get an explanation of what a budget is. 

A number of parameters of the game are fixed, for instance the fact that you need 
to make a budget, and that you have to deal with a number of economic events each 
month (invoices, neighbors who pay visits to borrow money, etc.). Depending on 
your choices, such as accepting a subscription offer from a magazine, or lending 
money to a friend, new things will occur the following month. 

The game contains links, marked with an i, to various sorts of information, and the 
player is explicitly told to click on these links for information. By using a “noteboard” 
in the flat, the player can go back to the general information given before starting the 
game. 

The player thus has to start up making a budget (Fig. 13.2), where the steady 
sources of income are already there, while the player can choose how much to 
spend on consumable goods (Sw. férbrukningsvaror), such as food, clothes, and 
hygiene, and how much to save. On this page, an explanation of “surplus money” 
(Sw. överskott) is given if you click i. 
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Fig. 13.2 A budget in the Economy Game (UngaFakta 2020, printed with permission from 
UngaFakta) 


Apart from the fixed gaming parameters mentioned above, a number of economic 
parameters are also fixed in this game. Examples are the sources of monthly income, 
such as student grants and maintenance, which appear to be in line with the Swedish 
terms. The cost of rent is also fixed. Therefore, the player cannot choose a cheaper 
way of living, such as renting a room or sharing a flat with others. 

Given the fact that this game is supposed to be used to learn about private economy, 
itis somewhat surprising that no information is given about reasonable costs for food, 
consumable goods, or the like. In a game like this, links to other websites with that 
kind of information could have been included. Also, one could argue that choices such 
as “no fast food or restaurants” or “no unnecessary costs for clothes this month” would 
be available (or the opposite, to see the consequences of excessive consumption). If 
students are using a game like this, it would be wise to supplement the game with 
classroom discussions and collaborative learning activities, for instance to search for 
useful web resources about private economy, or what standard of living is suggested 
as reasonable. 

Also, it is astounding that this game gives you a warning if you have spent too 
little on clothes, and that you therefore are running a risk that people around you 
“might start complaining” (see Sect. ). Furthermore, in the game you can only 
plan for such costs in the budget, while you cannot decide when to go out to buy your 
soap or your clothes, or what to buy. With regard to furniture, and technical products 
such as computers and television, you decide for yourself when to buy things, and 
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Fig. 13.3 Furniture shop in the Economy Game (UngaFakta 2020, printed with permission from 
UngaFakta) 
how much to spend (there are different price categories) (Fig. 13.3). Though you 


cannot choose to buy things second hand to keep your costs low. However, if you get 
into financial trouble, you can decide to sell a piece of furniture and get a refund of 
about half of the price. 


13.1.2 Interaction Between Text Resources 


Figure | shows the flat the player lives in, and in the lower part of the screen 
a number of symbols for game episodes automatically generated by the game. The 
symbols are more or less conventionalized, combined with different coloring. A 
green dollar sign means incomes, a red one for invoices that need to be taken care 
of. The color coding is equivalent to that of calculation software, in which negative 
numbers are displayed in red. A letter symbol is used for episodes that you need to 
take care of at a more or less daily basis, like offers from magazines, or for phone 


subscriptions. 
When clicking on triangles with exclamation marks, you can read that you have 
spent too little on food, clothes or hygiene (Fig. 13.5)—though as mentioned above, 


what is a reasonable budget for that is not indicated. Beside the dollar sign in the 


13.1 The Economy Game 


Fig. 13.4 Layout in the Economy Game (UngaFakta , printed with permission from 


UngaFakta) 


Low level of 
hygiene 


You have spent 
very little on 
hygiene articles 
the past month. If 
you do not do 
something about 
it, there is a risk 
that people 
around you will 
start complaining. 


Fig. 13.5 Hygiene warning in the Economy Game (UngaFakta 0, printed with permission from 


UngaFakta) 
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upper right corner, you can see how much money you have got. The dollar sign is 
thus used as a general symbol for economy, though you cannot choose currency in 
your budget (instead Swedish crowns, SEK, appear to be default). 


13.1.3 Values 


The values given through the game is only commented on briefly. An obvious one is 
of course the whole idea behind the game: that it is important to be able to make a 
budget, and to have your private economy under control. 

However, there are other values in the game as well, some of which have been 
commented on above. One such example is that the game warns you when you have 
spent too little on, for instance, clothes. Another value given in the game is that you 
should help your friends. If you agree to lend some money to a friend (a boy) who 
is lagging behind with his rent, the next month you will get back the same amount 
of money plus an extra sum, as a thank you from your friend, who happens to have 
won some money on lottery, and wants to share it with you as thanks for the help. 

Since a number of implicit values are built into this game, it can be fruitful to 
let the students reflect upon them and discuss them in class, not least since some of 
these values are more or less in conflict with the general discourse about spending 
today. 


13.2 MinecraftEdu 


Minecraft was originally launched in 2011. This game seems to be very engaging, and 
it gives the player an opportunity to build things by using different kinds of (virtual) 
blocks. It supports creativity as well as the testing of different solutions, and it gives 
a feeling of mastery through immediate and relevant feedback. The learning process 
can be both individual and collaborative. The game appears in different “modes”: 
in the “survival mode” or “hardcore mode” of the game, you can build tools to kill 
animals or monsters, but in the “creative mode”, you can concentrate on building 
things—such as a piano or parts of the body, or houses and communities. This mode 
is the one that is mostly used in school contexts. 

MinecraftEdu (also called “vanilla” Minecraft) is an adapted version of Minecraft 
for educational use. However, in both versions you can set up a “joint world” (using 
the local area network—LAN) for teachers and students. In fact, many teachers seem 
to still use the Minecraft version and not MinecraftEdu. You can play Minecraft 
and MinecraftEdu as a single player, but it also supports collaborative learning and 
communication, for example in a class. 

MinecraftEdu can be used in different school subjects, for example to explore 
content and locations in history lessons or to construct maps to navigate in a (virtual) 


13.2 MinecraftEdu 127 


environment. In addition, you can explore biological mechanisms or train mathe- 
matical skills, etc. It is also possible to support transdisciplinary learning, which is 
enhanced under the label of “21st Century Skills” (or “Competences”’), as well as to 
support pupils with special educational needs. 

In such games, the students have to gather information or build something up. As 
a teacher, you can also add tasks, example the possibility for the students to write, 
so that the students can practice writing along with their gaming, and thereby benefit 
from the joint engagement in the classroom. However, it is important to make the 
learning goals and objects of learning explicit to the students. 

Minecraft is but one example of on-line resources that could be used in education. 
The reason for mentioning it here is that it is an example of a game that includes the 
possibility to add chats and other sites for communication and reflection. 


13.3 Using the Model for On-Line Gaming 


As is evident from our analyses of digital websites and educational games, our model 
for working with multimodal texts can be utilized for various online resources as 
well. However, since not all digitized material is produced for educational purposes, 
the use of such material primarily require answers to the question of educational 
relevance. Also, such use requires a reorientation of traditions and routines in educa- 
tional planning, as well as in existing cultures of recognition and testing practices 
(Kress and Selander 2012; Selander 2008, 2015). In gaming, the player will show 
his or her understanding through acting within the frames of the game or simulation. 
Therefore, there is a need for new ways of tracking discussions between students, 
as well as decisions made in the game over time. To conclude, by using digitized 
tools and environments, the range of ways to show learning and knowledge (‘signs 
of learning’) changes. In the case of collaborative problem solving, our model could 
help both to guide the construction of the “testing” situations, and the interpretation 
of the results. With this said, with minor adjustments our model for working with 
multimodal texts can be a useful tool for teachers who want to use online resources 
in education in meaningful ways. 

A starting point for working with the model is to analyze the general structure of 
the text, including such aspects as thematic orientation and sequencing, followed by 
the interaction between textual parts. However, when using gaming and other online 
resources in education, it seems relevant to also look at the student’s interaction with 
different parts of the text (e.g. the game), both in terms of what possibilities there 
are in the game and in terms of choices made by the student. 

As regards figurative language, the games can be scrutinized for example by 
looking at what domains the game or simulation uses for the exercises in mathe- 
matical, historical or other subject domains. Here it is relevant to discuss with the 
students for instance to what extent the game builds on analogies between the real 
world and the imaginative world of the game, and to discuss possible limits with the 
choices made by the constructors of the game. 


128 13 Educational Games and Online Resources 


Concerning explicit and implicit values, it is important to look into what kind of 
actions that are requested or possible in the game, and to discuss these in relation to 
values with the students. Another aspect of value in relation to gaming —in fact in 
relation to any learning activity—concerns what kind of learning the game, or activity, 
contributes with. Even though the use of games in education can be engaging for 
students, that is not an end in itself. Instead, all learning resources must be used for 
a specific purpose within the frames of the educational setting. 
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Chapter 14 A) 
Summary of PART II get 


Based on the model presented here for multimodal text analysis, we have analyzed 
a number of texts from a variety of subjects. The examples have mainly been taken 
from printed and digital textbooks, but we have also commented on websites and 
educational games for young learners. As regards the educational perspective, we 
have also commented on aspects of the texts that teachers and students can discuss in 
relation to different aspects of the model. Such discussions can be organized either 
through classroom discussions, or by way of digital media. Blogs can be utilized, 
as well as web-based resources enabling anonymity among the users, which can 
encourage more voices to come through in the collective learning process. What 
choices to make always depends on the student’s age and his or her prior experience 
of using digital media for similar purposes. 

The outset of the above analyses is the general structure and setting of the texts. 
Here we have commented on aspects such as sequencing and thematic orientation. 
The aim of starting with that is for students and teachers to get an overall impression 
of the role of different visual resources and what they offer the reader. With regard to 
the educational perspective in this part, we have tried to point out aspects of the texts 
that teachers and students can pay special attention to in the classroom, especially 
regarding potential challenges in the specific texts. 

As a second step, we have elucidated the interaction between parts of the text. Such 
interaction can concern visual proximity and congruence between the verbal text and 
other semiotic resources, but also the congruence between, for instance, the use of 
concepts in descriptions and explanations. As regards the educational perspective, 
teachers and students can reflect upon the interaction between resources, and what 
view of the knowledge area appears to be central in the text, and furthermore to 
clarify aspects that might be unclear or even lacking in the text for the content to 
become clear. 

As a third step of the analysis, we have focused on figurative language, given both 
through verbal language and through other resources. From the educational perspec- 
tive, teachers and students can “break up” the content of the figurative language and, 
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for instance, discuss the reach of analogies and how well they function in relation to 
the content in question. 

The final step of the analysis deals with explicit and implicit values. Implicit 
values can for instance be given through metaphors or in illustrations, or through the 
choice of content. In relation to the educational perspective, teachers and students 
can discuss dichotomies such as right or wrong, us or them, male or female, explicit 
or implicit, since such dichotomies, whether in combinations or singly, can reveal 
underlying values. 

We would like to stress two things in particular. First, multimodal analyses can be 
an aid for teachers to reveal a number of specific features of the text, and this can be 
a basis for classroom discussions with the students. That, however, does not mean 
that the teacher needs to point out all the different aspects in every single multimodal 
analysis. Second, the teacher can always decide to attach more or less weight to the 
different aspects when analyzing the texts. For example, when working with a topic 
such as attractive force in the physics classroom, discussing values might not be as 
important or interesting as it would be in history or religion. 

Also, we want to point out that close readings of text quite often lead to expositions 
of potential challenges or even discoveries of “flaws” in the text. We would claim, 
however, that it is almost impossible to write a textbook that works perfectly for 
students without giving them some kind of scaffolding. Therefore, our main aim 
in developing our model, and in scrutinizing texts with the help of it, is to provide 
teachers and students with tools that can be helpful in various text encounters. Also, 
when students have developed their knowledge about texts and multimodality at a 
general level, this knowledge can be used to cope with a variety of textual challenges. 


14.1 Texts and Multimodal Text Focus in Different Content 
Areas 


In some content areas, highly abstract or complex content needs to be concretized, 
such as the digestive system, chemical bonds in science, or political systems and 
aspects of social security in social sciences or economics. Quite often, seemingly 
simple texts contain a number of potential challenges. This in combination with 
unclear connections between verbal text and illustrations can lead to unnecessary 
challenges for the novice in the field. In each text encounter, students need to be 
aware of the ways in which all semiotic resources, in themselves or in combinations, 
may carry important meaning. 

In our model, special attention has been paid to figurative language. Within the 
science area, metaphors are part of the disciplinary discourse. At the same time, 
different forms of figurative language are used for pedagogical reasons to connect to 
students’ everyday life, or to visualize abstract content. Since figurative usages are 
so affluent, there are good reasons to discuss them and to “unpack” them to discuss 
their “reach”. Such discussions can also be enlightening for the teacher, who can 
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then see what ideas they lead to among the students. This is important, not least from 
the perspective of second language learning. 

Social sciences, too, use an abundance of metaphors. Here the question of values 
becomes especially important. Some values are clearly and openly expressed, while 
others are more hidden in writing and illustrations. One way of making values 
visible is to make comparisons. Especially dichotomies like us/them, female/male, 
center/periphery can make visible what is implied in the text. Another aspect of 
values in texts is the fact that the choice of illustrations can encompass stereotypes 
in ways that might not be as obvious as they would be if given through words. 

When analyzing digital texts it is perhaps even more evident what a multimodal 
analysis can do. Such texts often contain numerous animated or still visual illus- 
trations of various kinds, sometimes in connection with audio clips. Since digital 
texts often include links, there is no clear reading order, and in the case of external 
links there might be uncertainty about what is actually “the text”. The use of digital 
games has also become more common in teaching and learning. In these cases, it is 
important to scrutinize the content and what you can actually learn from the game, 
with or without other resources. As we have shown in analyses above, our model can 
also be a tool for assessing the learning potentials of games. 

A number of subject areas have components of “doing” apart from the more theo- 
retical aspects of the content. This accounts for parts of mathematics, experimental 
work in the science area, the critical discussions of source material in history or exer- 
cises in producing different genres in mother tongue subjects. Some school subjects 
also have a clear emphasis on training practical skills, such as music and design, 
crafts, or vocational programs. 


14.2 Multimodal Texts and the Future of Learning 


Even though we use a couple of examples from pedagogical games and an Internet 
page used for the early school years, we have mainly focused on printed textbooks 
or textbooks that have been published in two versions: one paper-based, one digital. 

However, there are good reasons to be open to the challenges associated with 
digital techniques such as mobile learning, the flipped classroom, video commu- 
nication, simulations, and game-based learning. With texts used in such settings, 
multimodal aspects are vital for information and knowledge processing as well as 
for knowledge representation. 

We hope that a greater and more systematic focus on multimodal aspects of texts, 
where our model can be one important tool, can be one way of dealing with the new 
text landscape associated with the digitization of education. We also hope that our 
way of tackling texts can make teachers feel confident when working with multimodal 
texts in these new domains, with regard to (i) the evaluation of content and structure 
in text that students encounter in their learning process, (ii) cultivating students’ 
multimodal text production, and (iii) the assessment of students’ multimodal texts. 
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14.3 Afterword 


Our model was developed for analyzing and working with texts such as the ones 
analysed in Part II of the book. Since it was developed in relation to multimodal 
texts, from an extended text concept, it has later been adapted and used in wider 
contexts. One example is an analysis of a physics lesson, where the lesson itself 
including activities and resources used was regarded “text” (Danielsson 2017). In 
another study the model was used to analyse museum exhibitions (Insulander 2019). 
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